Human subjects requirements and economic education researchers.

Lopus, Jane S. ; Grimes, Paul W. ; Becker, William E. 等

I. Introduction

All academic institutions that receive funding from the United States federal government are required to enforce regulations that govern the use of human subjects by their researchers. When a study meets the government's definition of research, the principal investigator must submit a proposal outlining the methodology and procedures to an internal Institutional Review Board (IRB) prior to engaging in any research project that includes collecting and/or analyzing data from human subjects. The local IRB must certify that the design is in conformity with the federal regulations before any research project using human subjects may begin. Thus, virtually all university professors in the U.S. who use their students for research into the scholarship of teaching and learning must be familiar with the IRB regulations and practices.

Although many classroom-based educational projects either do not meet the regulation's definition of research or are explicitly exempted from the human subjects protocols, the IRB system may still impose significant costs on project directors. The underlying rationale for the federal regulations and the IRB process is to protect human subjects from potential harm that may result as a consequence of participating in a research project. For some types of research studies, such as medical drug trials, the personal risks may be obvious and potentially serious. However, for classroom-based studies that normally rely on surveys and tests, the risks of personal harm are minimal or non-existent.

In recent years, a small number of high profile cases where careless procedures were employed in medical studies caused universities to tighten their oversight of all human subjects research. These cases included the deaths of two research volunteers, one at the University of Pennsylvania and another at Johns Hopkins University (Brainard 2005). The increased scrutiny and the burdens of conforming to the IRB policies led to public complaints by social scientists and educators whose typical research procedures do not pose significant risks to their human subjects. The 2002 Annual Report of The Journal of Economic Education (Becker 2002) calls into question the necessity of imposing the same IRB regulations that protect human subjects in medical studies to the area of classroom teaching. Risk-averse university officials, observing lawsuits filed against universities for human subjects violations in medical experiments, may "overreact when confronted with human subject committee members' arguments to expand their policing function to classroom teaching." It is argued that overly stringent IRB requirements create unnecessary burdens and hurdles for economic education researchers, and thus, less classroom-based research will be conducted. (1)

This paper investigates the extent of knowledge held by economic education researchers about the federal regulations that govern human subjects research, the perceived costs of these regulations, and whether the regulations significantly affect the quantity and quality of research done in economic education. Our analysis is based on information obtained through a web-based survey directed to those who recently conducted and published research in economic education and those likely to do so. After a brief background review of human subjects' protocols in social science research, we will discuss the survey results and the implications of our findings.

II. Background

The current federal regulations that govern human subjects research evolved from the recommendations of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research which was formed by Congress in 1974. This commission's final report, released in 1979 and popularly known as the "Belmont Report," (2) identified and defined the basic ethical principles on which today's regulations are based. The commission categorized these principles into the following three areas: 1) Respect for persons: To ensure the honor for the personal dignity, autonomy, and right to privacy of individual human subjects. 2) Beneficence: The obligation to minimize the risks of potential harm to human subjects while seeking to maximize the benefits of research to humanity. 3) Justice: To ensure that all benefits and costs of human subjects research are fairly and equitably distributed. These principles serve as the foundation for the Federal Policy for the Protection of Human Subjects (Code of Federal Regulations Title 45-Part 46) which institutionalizes the IRB process. Currently, seventeen federal departments and agencies that support and conduct human subjects research enforce this policy, which is often referred to as the Common Rule. (3)

The specific policies of the Common Rule are extensive but surround a small set of key issues. Primary among these issues is the requirement to obtain informed consent from all experimental participants in a research project. All human subjects must be free to both volunteer and withdraw from participation. Researchers are required to determine the potential risks, both physical and mental, that may result from participation in a project and to inform all human subjects about these risks prior to requesting their consent to participate. Furthermore, researchers are required to estimate all of the potential benefits and costs of the research project and to equitably select human subjects from the pool of individuals most suitable for the research questions being asked. The Common Rule also contains policies specifically designed to protect the rights of children and prisoners. While the regulations establish the roles and responsibilities of the local IRBs, they also provide institutions with a substantial degree of flexibility in how the boards are organized and operated.

If an academic institution receives federal financial support for any purpose, all investigative projects conducted at that institution must adhere to the Common Rule policies whenever human subjects are involved. Two major caveats to this rule exist. First, a project must meet the Common Rule's definition of "research" in order to be subject to the IRB process. Research is defined as "an activity designed to test an hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge." (Note that this definition does not rely on whether a project's results are intended for public dissemination or for personal consumption.) The regulations allow either the investigator or the local IRB to make the determination as to whether a project meets this definition. In practice, this can lead to confusing results. The survey project presented here provides a classic example; one of the author team's institutions accepted an author's statement that presentation of descriptive survey results was not generalizable analysis and, therefore, the project did not meet the definition of research. The institution of another author conducted an IRB review that concluded the project was research but in conformity with the appropriate human subject protocols. The third institution's IRB also concluded that the project was research but ruled that it was exempt from the human subjects protocols and thus did not evaluate the project's conformity with the rules. (Three institutions and three different evaluation outcomes!) Given that different decision-makers may rule differently on whether a project meets the Common Rule's definition of research, rational project directors seeking to minimize their own personal risk will naturally make an IRB application in marginal cases. Thus, even though a project may be outside the jurisdiction of the local IRB and thereby be explicitly excluded from having to meet the human subject protocols, in practice significant time costs may be incurred by the project director in order to secure an official waiver for exclusion.

Second, the Common Rule specifically states that studies undertaken in an established educational setting using normal educational practices (including tests) are "exempted" from the policies. However, individual investigators are not free to exclude their own research projects from IRB oversight; only the local IRB is allowed to exclude research projects for meeting the definition for exempted educational studies. Many local IRBs routinely exclude most classroom-based educational projects from having to meet the Common Rule regulations. In some cases, IRBs have excluded whole classes of academic practices, most notably oral history projects, from their purview (Brainard 2003). In other cases, fierce battles have raged between project directors and local IRBs over what constitutes normal educational research (Brainard 2004; Howard 2006). Regardless of local precedents, all project directors at most American universities still bear the burden of completing the paperwork associated with an IRB application in order to determine if a classroom-based educational research study will be declared exempt.

A number of authors have recognized the inherent conflict between researchers and IRBs. Howe and Dougherty (1993) note that researchers naturally feel that they are in a better position to evaluate the ethics of their own research practices as opposed to an IRB that is often populated with scientists and administrators from other specialized fields. This is particularly true for social scientists who confront IRBs where the physical sciences are often heavily represented. Oakes (2002) calls on social scientists to accept the legitimacy of IRB oversight, to learn more about human subjects regulations, and to educate IRB members about social science methodologies and practices. It has also been argued that IRBs at research universities are often overloaded with work and may be inadequately prepared for their assigned tasks. Noting this, Pritchard (2002) calls for an increase in resources to adequately prepare IRB members and to educate researchers in their responsibilities for meeting the human subjects regulations.

Interestingly, to date only one major empirical study has appeared on the effects of the IRB process on research activity. Gray, Cooke and Tannenbaum (1978) found that both researchers and IRB members were generally supportive of the system and believed that it did protect the rights of human subjects. They also found that the average IRB required revisions and modifications to about half of the proposals it received for review. Only ten percent of the researchers surveyed thought that IRBs impeded research, made decisions from an unqualified position, and that the costs of the system exceeded its benefits. However, it should be noted that Gray, Cooke and Tannenbaum's (1978, 1095) sample consisted primarily of medical schools, hospitals, and other medical-related institutions, and that fewer than 7% of the studies reviewed involved educational innovations. Furthermore, it should be noted that these results are more than twenty-five years old and reflect a time before today's Common Rule had been adopted by agencies outside what was then the United States Department of Health, Education and Welfare.

In today's regulatory environment, research economists routinely come into contact with the IRB process. This usually occurs when an economist proposes to use a secondary database, such as a Census survey, containing information that has the potential to identify individual respondents. In such cases, the data have been collected by a third party and the researcher must only demonstrate that the research design will not result in the public revelation of confidential private information or personal identity. Economic educators and experimental economists who collect primary data directly from people, in many cases their own students, have a greater burden to prove that their practices meet the federal regulations or are exempt from them. Friedman and Sunder (1994) discuss the IRB process from the experimental economists' perspective and note that some university IRBs have granted blanket approvals or exemptions to research using experimental economics techniques. However, Friedman and Sunder also point out that ethical dilemmas may arise in classroom settings if the researcher is not careful in his or her project design. For example, using grade incentives to motivate students into participating in an experiment may cause conflicts between pedagogical goals and research goals.

Even after recognizing that the Common Rule and the IRB process it mandates play important roles in the self-governance of research practices at American institutions, the question still remains as to their effect on classroom-based research practices that rely on surveys, tests, and classroom experiments, and thus, present little if any significant potential to harm human subjects. The recent trend of tightened IRB oversight may have unintended negative consequences by imposing regulatory burdens that discourage researchers. Our survey was conducted to determine what economists know about the federal human subjects regulations and to determine if the regulations have an impact on the quantity of primary human subjects research undertaken.

III. Data Collection and Results

To investigate how the mandated Common Rule regulations affect economic education research, we designed a 39-question web-based survey instrument. The questions asked respondents about their institutions' local IRB human subjects procedures and how those procedures affected the research of the respondent. Background information was collected about the respondent's gender, university position, work-time allocations, and the amount of research he or she conducted and published. The complete survey can be found at: (4) http://misweb. cbi.msstate.edu/pgrimes/surveyIRB/

During the Spring of 2004, a solicitation email with a hot link to the survey was sent to all of those who had published articles in The Journal of Economic Education during the previous five calendar years, to those who had presented and discussed papers in the economic education sessions (5) at the annual meetings of the Allied Social Science Association from 1997 through 2004, to all subscribers of the NAEENET and TEACHECON listservs, (6) and to the Center and Council directors of the National Council on Economic Education network. Although some people on these list serves and in the NCEE network are not researchers, the cover statement to the survey solicited responses from those "likely to have conducted economic education research involving human subjects." A follow-up request for responses was sent during the summer of 2004. There were 110 responses to the survey. (7)

Descriptive statistics for the respondents' demographic and background characteristics are reported in Table 1. The sample approximates the current gender mix of Ph.D. graduates in economics, with approximately 75 percent being male and 25 percent being female (Siegfried and Stock 2004). The respondents were fairly evenly divided between full and associate professors but a smaller percentage of the sample reported working at the assistant professor or lower rank. Although the respondents reported more time being spent on teaching, most respondents also reported a significant amount of time devoted to research. Almost a quarter of the sample reported spending more than 40 percent of work time on research activities. More than half of the sample worked in masters and doctoral granting departments while about 38 percent worked in departments that only grant the bachelor degree. Overall, there is nothing in the descriptive statistics to suggest that this sample is not drawn from the normal distribution of economics professors specializing in economic education research in American academe.

As expected, the survey respondents were generally experienced with research involving human subjects. This is apparent in the responses to the queries reported in Table 2. More than 60 percent of the sample reported conducting human subjects research over the previous five years, and almost all respondents had used students as research subjects during that time. Almost half of the respondents classified at least some of their research as being classroom experiments. The respondents were generally research active with about 83 percent publishing in refereed academic journals during the previous five years. (Note that the question concerning published research was not constrained to economic education studies with human subjects.)

Even though most of the sample had professional experience conducting human subjects research, very few were sufficiently knowledgeable to identify key definitions and policies within the regulations. Only 19 percent correctly identified the Common Rule's definition of "research" as noted above and only about 11 percent knew what qualified as "exempt research." Furthermore, only 19 percent knew that the Family Education and Privacy Act (the Buckley Amendment) enables teachers to have access to student information for the purpose of improving instruction.

Table 3 reports the frequency responses to a variety of survey questions that addressed the respondents' knowledge and experience with IRBs. About 91 percent of the sample reported that their employing institution maintained a standing IRB. Only about 5 percent reported that they did not know whether their institutions did so or not. Presumably, the remaining small percentage of respondents worked for institutions that do not accept federal funds or institutions not in compliance with the Common Rule regulations.

Given that the Common Rule allows institutions a degree of flexibility in the structure and operating policies for local IRBs, several survey questions addressed local IRB procedures and practices. Nearly 71 percent of the respondents indicated that their IRB reviewed classroom studies and experiments. Only about 12 percent indicated that classroom studies and experiments were not reviewed by their local IRB. This suggests that only a small minority of IRBs provided a blanket exemption to classroom-based research practices.

A substantial variation across institutions appears to exist in their methods of instructing professors about the regulations governing human subjects research. Collectively, about 44 percent of institutions represented in the sample required professors to attain a locally provided certification prior to submitting proposals to the IRB. These certifications signify that the professors know and understand the mandated human subjects protocols. The survey results suggest that about 12 percent of institutions employed test-based processes and a like number required workshop-based processes. As seen in Table 3, another 12 percent required both completion of a workshop and passage of an exam for certification.

When asked about recent interactions with their local IRB, about 55 percent indicated that they had submitted proposals for IRB review within the previous two years. A majority indicated that they had submitted five or fewer IRB applications over that time frame. With respect to the burden of IRB paperwork, 17 percent of the respondents indicated that it took an hour or less of their work time to complete an IRB application packet. The most common response to the question of time was "a few hours" with about 30 percent so responding. However, nearly a quarter of the respondents indicated that IRB paperwork took one or more days to complete. (8) Clearly there is a positive and substantial opportunity cost of researchers' time associated with the IRB process and the paperwork burden varies across institutions. It is important to note that there is no uniform or standardized IRB approval request form and that the amount of detailed information required varies dramatically from one institution to another. (9)

The survey also asked respondents about their experience in turnaround time after an IRB application was submitted. As seen in Table 3, about 20 percent of respondents had a final decision within only 1 to 5 days of submission. However, fully one quarter of the respondents indicated that a decision took between 16 to 31 days and another 14 percent reported that it took more than a month to receive a final decision from their local IRB. Again, the data suggest that a potential opportunity cost in lost research time exists due to the review process mandated by the Common Rule regulations.

Of central importance to this study is how IRBs and mandated human subjects protocols affect researcher behavior. Table 4 provides several interesting insights into this issue. Slightly more than 65 percent of the respondents had not significantly modified a proposed research project based on feedback from their local IRB and only nine percent reported that they had done so. For classroom research involving tests, surveys, and experiments, it is unlikely that these modifications involved significant ethical issues for the student subjects. Furthermore, a vast majority, 93 percent, had never canceled a research project based on a negative IRB review. These findings suggest that only in a minority of cases does IRB feedback result in significant changes in the project design or the procedures used by economic education researchers. (10)

Table 4 also shows that only 6 percent of the respondents believed that the quality of their research had improved due to the human subjects regulations. Additionally, over 23 percent reported that the enforcement of human subjects protocols presents a significant barrier to research, and another 19 percent were unsure whether it does or not. Close to 18 percent of the respondents reported that the enforcement of human subjects' protocols had reduced the frequency with which they conduct research projects using students, with 13 percent being unsure. Taken together, these survey results lead to the conclusion that the IRB process was more likely to produce discouraging barriers rather than to improve the quality of the research. However, on the positive side, 64 percent of the respondents indicated that they had not reduced the level of their research output due to the requirements imposed by the IRB process.

Table 5 reports the response frequencies for three questions regarding the professional review of publications resulting from research projects involving human subjects. Although not required by the regulations, only 22 percent of the respondents indicated that they explicitly noted in their professional writings that the mandated human subject protocols were followed. Less than 5 percent recalled a journal editor or referee questioning them about the procedures they followed to protect human subjects. None of the respondents had ever had a paper rejected for publication because human subjects protocols were not followed. At least two possibilities exist for these findings. First, journal editors and referees may trust that the IRB process is being followed and that the responsibility to ensure adherence to the rules lies with the authors' employing institution. Or, alternatively, editors and referees may believe that the human subjects regulations are not an important element of the research process and thus do not concern themselves with them when evaluating the merits of a research article.

IV. Conclusions and Recommendations

In recent years, the increased emphasis on enforcement of federal regulations that govern academic research involving human subjects resulted in a small but vocal outcry by social scientists who employ methodologies that impose little or no risk of harm to those who participate in their studies. While acknowledging that there should be safeguards to ensure personal dignity, respect, informed consent, and the right to privacy for human subjects in classroom studies, it is argued that the same safe guards necessary to protect the subjects of medical experiments should not be imposed on surveys and classroom-based educational research projects. In the fall of 2006, this argument received national attention when the American Association of University Professors (AAUP) released a report calling for "research methodologies that consist entirely of collecting data by surveys, through interviews, or by observing behavior in public places to be completely exempt from review by campus IRBs, and that there be no requirement of IRB approval for the exemption" (Thomson, Elgin, et al 2006).

The perceptions of economic educators revealed in our survey results tend to support this recommendation or at least suggest that modifications to streamline the current IRB process for classroom-based research should be seriously considered. However, given that it is highly unlikely that the Common Rule will be modified in the near term in accordance with the AAUP's recommendations (recall that seventeen federal agencies and departments would have to negotiate new rules!), the current status quo will prevail into the foreseeable future. This means that even though the federal regulations explicitly exempt most classroom-based educational research practices from having to meet the human subjects protocols, and exclude from IRB jurisdiction those projects not meeting the Common Rule's definition of "research," project directors will continue to submit research proposals to local IRBs to determine if a project is or is not subject to the protocols. Our survey results indicate that this practice may be costly in terms of the time that researchers must redirect to complete the regulatory paperwork and internal compliance process. Furthermore, this cost is not evenly distributed across researchers due to the latitude in interpretation that the Common Rule provides for local IRBs.

Given this, do our results provide any practical insights that may be useful for researchers contemplating a classroom-based project? Yes, there are at least three major points that we believe are important.

First, all economic education researchers should be thoroughly familiar with the Common Rule and the IRB process. Our results indicate that many researchers do not know or understand the prevailing definitions and rules as put forth in the federal regulations. This places researchers at a severe disadvantage. Only by understanding the regulations can researchers know when a project meets the Common Rule's definition of "research" or when to request that a project's classroom-based activity be declared exempt from IRB oversight.

Second, researchers should be thoroughly familiar with their local IRB policies and procedures. This includes whether or not they must become certified prior to conducting a project involving human subjects as well as how to submit a project proposal for review. Our results clearly indicate that practices vary from one institution to another and that local IRBs have discretionary powers that may result in different outcomes across institutions. A working knowledge of the local "home rules" reduces the time cost for researchers negotiating the IRB approval process.

Third, researchers should recognize that the current regulatory environment may impose an opportunity cost on their time and adjust their choices accordingly. For some, it may be possible to reduce the cost of compliance by acknowledging its existence and factoring it in when scheduling new projects. Knowing that it will take several days to several weeks for an IRB to review a project, researchers may be able to reallocate their professional efforts during that time in ways to minimize the cost. Our survey results indicate that the time required for an IRB to issue a final decision can vary dramatically. Thus, proper advance planning prior to initiating a project is crucial to avoid wasted time.

By considering these three points, researchers may reach a better understanding of the ethical issues involved in human subjects research and encounter fewer frustrations with their local IRB. However, a careful analysis of the overall costs and benefits of the current regulatory scheme appears warranted. In the long-run, if the federal regulations were modified to allow blanket exemptions for classroom-based research so long as dignity, respect, privacy, and informed consent were ensured, the result could be more research and, therefore, more knowledge on what works in economic education.

References

Becker, William E. 2002. 2002 Annual Report. The Journal of Economic Education. Available from: www.indiana.edu/~econed/anrpts/anrpt02/rep02 .htm

Brainard, Jeffrey. 2005. An Accrediting Group for Protecting Human Research Subjects Will Close. The Chronicle of Higher Education, Today's News, 13 September. Available from: http:// chronicle.com/daily/2005/09/2005091304n.htm

Brainard, Jeffrey. 2004. When is Research Really Research? The Chronicle of Higher Education 51 (14): A21.

Brainard, Jeffrey. 2003. Federal Agency Says Oral History Is Not subject to Rules on Human Research Volunteers. The Chronicle of Higher Education 50 (10): A25.

Center for Advanced Study. 2005. Improving the System for Protecting Human Subjects: Counteracting IRB 'Mission Creep.' The University of Illinois, Urbana-Champaign, IL. Available from: http://www.law.uiuc.edu/conferences/ whitepaper/

Friedman, Daniel and Shyam Sunder. 1994. Experimental Methods: A Primer for Economists. Cambridge University Press, New York, NY.

Gray, B.H., Cooke, R.A., and Tannenbaum, A.S. 1978. Research Involving Human Subjects. Science 201 (4361): 1094-1101.

Hamburger, Phillip. 2004. The New Censorship: Institutional Review Boards. Supreme Court Review 271 (2004): 271-354.

Howard, Jennifer. 2006. Oral History Under Review. The Chronicle of Higher Education 53 (12): A14.

Howe, K.R. and Dougherty, K.C. 1993. Ethics, Institutional Review Boards, and the Changing Face of Educational Research. Educational Researcher 22 (9): 16-21.

Oakes, J.M. 2002. Risks and Wrongs in Social Science Research: An Evaluator's Guide to the IRB. Evaluation Review 26 (5): 443-79.

Pritchard, I.A. 2002. Travelers and Trolls: Practitioner Research and Institutional Review Boards. Educational Researcher 31 (3): 3-13.

Siegfried, J. J. and Stock, W. A. 2004. The Market for New Ph.D. Economists in 2002. American Economic Review 94 (2): 272-85.

Thompson, J. T., Elgin, C., Hyman, D. A., Rubin, P. E., and Knight, J. 2006. Research on Human Subjects: Academic Freedom and the Institutional Review Board. Academe (92) 5: 95-100.

United States Department of Health, Education and Welfare. 2005. Code of Federal Regulations, Title 45, Public Welfare, Part 46, Protection of Human Subjects. Available from: http://www. hhs.gov/ohrp/

Footnotes

(1.) Becker's call to limit the scope of IRB involvement in the research process echoes similar sentiments across a number of disciplines. See for example the "Illinois White Paper" (The Center for Advanced Study 2005) for a discussion of the arguments against the perceived expansion of IRB "mission creep." Some scholars have gone so far as to decry that the IRB process violates academic freedom and that the federal regulations are unconstitutional (Hamburger 2004).

(2.) This title refers to the Belmont Conference Center at the Smithsonian Institute where the commission met.

(3.) The Common Rule has been adopted by the following: Department of Agriculture, Department of Commerce, Department of Defense, Department of Education, Department of Energy, Department of Health and Human Services, Department of Housing and Urban Development, Department of Justice, Department of Transportation, Department of Veterans Affairs, Agency for International Development, Central Intelligence Agency, Consumer Product Safety Commission, Environmental Protection Agency, National Aeronautics and Space Administration, National Science Foundation, and the Social Security Administration. Additional human subject regulations are also enforced by the Food and Drug Administration.

(4.) Because the survey was housed on the Mississippi State University computer system, our research design and practices were submitted for review to the MSU IRB. The application was submitted on July 2, 2003. Notification that an administrative review of the project revealed that it was in adherence with the Common Rule regulations was received on August 4, 2003. The MSU IRB also required official documentation from California State University and Indiana University that the project team members at those institutions were certified in human subjects research. For multi-institutional studies, local IRBs have the discretion to require that approval also be obtained from collaborators' institutions.

(5.) These research sessions were organized by the National Association of Economic Educators and the National Council on Economic Education in cooperation with the American Economic Association and the Allied Social Science Association.

(6.) NAEENET subscribers are members of the National Association of Economic Educators and others interested in economic education. TEACHECON is a listserv dedicated to issues surrounding the teaching of economics, primarily at the university level. Subscribers are economics professors from the entire cross-section of institutions of higher learning.

(7.) We are unable to report a response rate due to the dynamic and fluctuating nature of listserv subscriptions.

(8.) A closer examination of the results reveals that there is a natural "learning curve" to completing the IRB paperwork. Of the respondents who had completed five or more IRB approval applications, more than two-thirds reported that the paperwork took a few hours or less. For inexperienced respondents, those with less than five completed IRB approval applications, only forty-five percent so reported.

(9.) Typical application forms range from two or three pages to more than a dozen pages. The interested reader is referred to the University of California at Berkeley's "Protocol Narrative Form" as representing a typical example. A link to this form can be found on the World Wide Web at: http://rac.berkeley.edu/ compliancebook/print.html

(10.) Such a conclusion is in stark contrast to the findings of Gray, Cooke and Tannenbaum who found that in the late 1970s about half of all applications submitted to IRBs resulted in modifications to the proposed projects. However, Gray, Cooke, and Tannenbaum were primarily looking at medical-related research, which may explain the observed difference with our results.

Jane S. Lopus *, Paul W. Grimes, ** William E. Becker, *** Rodney A. Pearson ****

* California State University--East Bay; 510.885.3140; [email protected]

** Mississippi State University; 662.325.1987; [email protected]

*** Indiana University; 812.855.3577; [email protected]

**** Mississippi State University; 662.325.1995; [email protected]

The authors wish to thank all of the economic educators who responded to the survey on which this paper is based. Special thanks are extended to Claudia Parliament, Arthur Goldsmith, and the participants of the economic education research sessions at the 2006 American Economic Association Meetings and the 2005 Southern Economic Association meetings. Editorial assistance was provided by Marybeth Grimes.

TABLE 1
Characteristics of Survey Respondents

Characteristic Percent of Sample

Gender:
 Male 74.55
 Female 25.45
Academic rank:
 Full Professor 38.18
 Associate Professor 35.45
 Assistant Professor 20.91
 Instructor 3.64
 Other 1.82
Percentage of time spent teaching:
 0 2.73
 1-40 39.09
 41-80 55.45
 81-100 2.73
Percentage of time spent on research:
 0 0.00
 1-40 72.73
 41-80 20.91
 81-100 3.63
Percentage of time spent on service:
 0 4.55
 1-40 94.54
 41-100 0.91
Percentage of time spent on administration:
 0 54.55
 1-40 36.36
 41-80 7.27
 81-100 1.82
Highest degree offered by respondent's department:
 Doctorate 28.04
 Masters 27.10
 Bachelors 38.32
 Associate 0.93
 Other 0.93
 None 4.67

Number of Observations 110

Note: Non-responses to time allocation questions counted as zero.

TABLE 2
Survey Respondents' Experience with and Knowledge of Human
Subjects Research

Activity Percent of Sample

Number of research projects utilizing
human subjects completed in past 5 years:
 0 17.27
 1-5 63.64
 6-10 13.64
 More than 10 5.45

Percent of human subjects researchers
who have used students as subjects
in past 5 years: 93.55

Percent of human subjects researchers
who classify some of their work as
experimental economics: 41.05

Number of refereed articles published
in past 5 years: *
 0 17.27
 1 10.00
 2-5 44.55
 6-10 17.27
 11-20 10.00
 More than 20 0.91

Percent of human subjects researchers
who could identify the Common
Rule's definition of "research":
 Correctly 19.10
 Incorrectly 15.45
 Didn't Know 65.45

Percent of human subjects researchers
who could identify the Common Rule's
definition of "exempt research":
 Correctly 11.11
 Incorrectly 29.63
 Didn't Know 59.26

Percent of human subjects researchers
who could identify the Buckley
Amendment's rule on collecting student
information for instructional improvement:
 Correctly 18.86
 Incorrectly 16.98
 Didn't Know 64.16

Number of Observations 110

* Not limited to research involving human subjects.

TABLE 3
Survey Respondents' Knowledge of and Experience with Internal
Review Boards

Question Percent of Sample

Does your institution maintain a standing
Internal Review Board (IRB) to oversee
compliance with federally mandated human
subject protocols of research?
 Yes 90.91
 No 3.64
 Don't know 5.45

Does your institution require classroom
studies and experiments involving student
subjects to be reviewed by an IRB or
other oversight committee?
 Yes 70.91
 No 11.82
 Don't Know 17.27

Which of the following options best
describes your institution's requirement
for becoming certified to conduct a
project involving human subjects?
My university:
 Does not require certification 55.88
 Requires passing a test 11.76
 Requires completion of course/workshop 12.75
 Requires both taking a course/workshop and
 passing a test 11.76
 Requires either taking a course/workshop or
 passing a test 7.84

About how many studies involving student
subjects have you submitted for review by
your institution's IRB or oversight
committee during the past 2 years?
 None 44.54
 1-5 51.82
 More than 5 3.64

Based on your experience, how long does
it take on average to complete the
application and associated paperwork for
review?
 An hour or less 16.98
 A few hours 29.25
 About a day 12.26
 More than a day 12.26
 No experience with the process 29.25

Based on your experience, about how many
working days does it take on average to
receive a final decision from your IRB
or oversight committee after the
paperwork has been submitted?
 1-5 days 19.39
 6-10 days 18.37
 11-15 days 18.37
 16-31 days 24.48
 More than 31 days 14.29
 Don't know or N/A 5.10

Number of Observations 110

TABLE 4
Survey Respondents' Opinions on Internal Review Boards and Human
Subjects Protocols

Question Percent of Sample

Have you ever significantly modified a
research project based on feedback from
your IRB or oversight committee?
 Yes 9.35
 No 65.42
 N/A 25.23

In your opinion, does your institution's
enforcement of human subjects protocols
present a significant barrier to research
involving student subjects?
 Yes 23.36
 No 52.34
 Unsure 18.69
 N/A 5.61

Have you ever cancelled a project because
of a negative opinion by your IRB or
oversight committee?
(Indicate the number of projects that
you have cancelled for this reason.)
 None 93.07
 One 5.94
 Two 0.99

Has the enforcement of human subjects
protocols by your institution reduced
the frequency that you conduct research
projects using student subjects?
 Yes 17.92
 No 64.15
 Unsure 13.21
 N/A 4.72

In your opinion, has the quality of your
research improved because of mandated
human subjects protocols?
 Yes 5.66
 No 73.58
 Unsure 16.04
 N/A 4.72

Number of Observations 110

TABLE 5
Human Subjects Protocols and Publication Activity of Survey Respondents

Question Percent of Sample

When publishing your research based on data
collected from student subjects, do you
usually explicitly note in your articles that
human subjects protocols were followed?
 Yes 22.11
 No 60.58
 N/A-I do not follow human subjects protocols 17.31

Have journal editors or referees ever
questioned the procedures you used to
collect data from human subjects?
 Yes 4.81
 No 86.54
 I don't recall 8.65

Have you ever had a paper rejected for
publication or presentation because
appropriate human subjects protocols
were not followed?
 Yes 0.00
 No 93.33
 I don't know 6.67

Number of Observations 110