Research and Experimentation in Teaching Effectiveness:
The Ethical Review Process and the IRB
“The art of life is (i) to live, (ii) to live well, (iii) to live better.”
Alfred North Whitehead, The Function of Reason (1929)
The art of teaching is to teach, to teach well and to teach even better.
Philip A. Pecorino, Ph.D, Shannon Kincaid, Ph.D., Belle Gironda, Ph.D.;
Queensborough Community College, The City University of New York
Research and experimentation in teaching effectiveness, which makes research subjects of students in regular classroom settings, should be subject to an ethical review process that is separate from, and in addition to the mandated review of human subjects research that is conducted by Institutional Review Boards.
Furthermore, despite the moral dilemmas that often characterize ethical decisions; in the review of teaching effectiveness, we should not shy away from developing and applying a basic set of normative principles for ethical decision making.
Summary of Rationale
1. Review by an Institutional Review Board (IRB), as the sole form of review of pedagogic research, may provide insufficient protection of the human subjects involved who are mainly and usually students.
2. IRB review usually deals with some of the key issues to which students, as research subjects, may be particularly vulnerable, including coercion and violations of privacy; but IRBs are not designed or prepared to address the very real ethical dilemmas that arise when individual instructors experiment with teaching methods, and when their informal research into teaching effectiveness pose the potential for harm to students.
3. Students are a particularly vulnerable population because their success in school
depends partially on decisions made by teachers, who may also wish to conduct informal research on their teaching effectiveness, and may want their students to serve as subjects of that research. Nevertheless, not all informal pedagogic research is subject to IRB review.
4. Because of burgeoning movement in support of “action research” and “practitioner research,” and because of the very nature and definition of such research, a large percentage of teachers involved in research on pedagogy are not trained experts in the field of pedagogic research. However, they are interested in studying and improving their own practices and/or adding to the growing body of scholarship and knowledge about teaching.
5. There will be ethical dilemmas to be resolved by those conducting pedagogic research who are teachers because they are teacher/researchers and their dual roles present conflicts in interests that have the desire for obtaining new knowledge set against the requirement that their subjects/students be protected from avoidable harms.
6. Each of the above conditions amplifies the need for an ethical review process for pedagogic research and suggests the potential usefulness of applying normative ethical principles to provide necessary protections for students and guidance for faculty researchers.
The Need for Pedagogic Research
We have argued elsewhere (Pecorino and Kincaid, 2007) that the scholarship of teaching and learning (SOTL) is not an isolated slice of scholarly research, but that it is a fundamental obligation of all college and university professors. The relationship between the collective professional responsibilities of faculty and the discipline-based scholarship of individual scholars entails a duty to conduct pedagogical research. Regardless of discipline, all faculty members have a basic professional obligation to both maintain a familiarity with current research and scholarship in SOTL, and participate in pedagogical research. This ensues from their status as professional educators. Such professionals have a responsibility to contribute towards and further their profession
Ethical Issues Related to Pedagogic Research
When educators engage in research in teaching and learning effectiveness, whether on an informal or formal basis, ethical issues will arise. Recent trends in the publications on SOTL indicate that there a growing awareness of the ethical issues associated with turning classrooms into laboratories. Yet among the research on the ethics of SOTL, there is little discussion of what we propose to provide in this paper, namely a conceptual framework for ethical decision-making in both formal and informal research in teaching and learning. (Kincaid and Pecorino, 2006)
In providing a framework for ethical decision-making in the context of pedagogical experimentation and informal research in the classroom, we are making two fundamental assumptions. First, while the ultimate goal of such research is improved pedagogy, informal and formal research in teaching effectiveness holds the potential for harm to students, and must be subject to some basic ethical guidelines. Second, these guidelines must be rooted in the acceptance of education as a profession that has fiduciary responsibilities for learners and for students as subjects of pedagogic research.
Formal and Informal Pedagogic Research
We argue below that all post-secondary instructors have a professional obligation to engage in both formal and informal research intended to discover or produce more effective pedagogies. From something as simple as the selection of a new textbook to the most fully developed experimental design involving control groups, pedagogic experimentation and research takes place on a daily basis. Most teachers care about their students, and they want to do the best job they can in teaching their classes. Textbook choices are but one facet of the challenges of designing and implementing a class. The choice of a textbook is not a trivial thing: it must take into account the abilities of the teacher, the needs of the students, and the requirements of the course itself. If chosen haphazardly, a textbook can cause harm to student learners in many ways, including ineffective teaching, frustration with materials that are too advanced or too simplistic, cultural or political bias, all of which might result in a student coming away from a class with a negative reaction to the course and its content. So, even in the seemingly mundane process of selecting a textbook, we see that the two fundamental principles of the fiduciary relationship between teacher and leaner (First, do no harm, second, maximize pedagogical benefit) are operative.
Another potential source of potential harm and/or benefit for students lies in an instructor’s decisions regarding the use of technology. Any instructor contemplating the use of technology in the classroom (from web-sites to on-line instruction) must ask some serious ethical questions about their preparedness to effectively utilize these technologies, the effect these technologies can have on students (and educators) who are under prepared or unable to deal with the challenges of modern technology.
These examples represent just a few of many ethical challenges faced by individual instructors when they are fulfilling their professional obligations to improve teaching effectiveness beyond the scope of IRB review. But when an educator is engaged in formal research or is developing an educational research project, the same principles apply, even in the context of IRB approval. On these grounds, the IRB is seen as the institutional process which determines the acceptability of the research with human beings, and functions to protect the interests of the institution and the subjects. This process of ethical review (ERP), as based on the two principles outlined above, exists independently of the IRB, and they are not grounded in the institutional process of review. But what occurs when the review process exempts the research from review or a waiver is given? Is there no review of what is to be attempted with students? What occurs when the review process approves of the research: is the researcher now exempt from further review of the project? Are the ethical obligations and professional responsibilities of the educator carried out by the IRB? Do IRB’s absolve educators from their professional ethical concerns?
While a detailed description of the distinction between formal and informal research is beyond the scope of this discussion (See Pecorino and Kincaid, 2006), it is clear that some forms of pedagogic experimentation and research do not fall under the purview of IRB review. But, this does not mean that such proposals can forgo the IRB process entirely. In some institutions only an IRB, by reviewing a research proposal, can determine if the research qualifies as “Exempt,” which then means that it does not need to follow continuing review procedures unless there are changes in the research protocol.
Anecdotal evidence suggests that this proviso is variously interpreted and applied by colleges and universities and their IRBs, and could lead, in some cases, to relaxed standards or less rigorous review of pedagogic research involving human subjects. For this reason and others, including the vulnerability of students as a population, the fiduciary relationship between faculty members and the students they teach and the likelihood that some pedagogic researchers lack primary expertise in this type of research, we would like to propose the usefulness and need for an ethical review process (ERP) for pedagogical research, separate from and in addition to the IRB.
As the rapidly growing literature on action/practitioner research and pedagogic ethics clearly shows, pedagogic ethics has become a central element in the debates over effective teaching. Leading scholars and proponents of SOTL, (including the Carnegie Institute and others) have recognized the importance of and need for an ethical review process for informal pedagogic research. Our proposal is to extend this conversation to include the recognition and embrace of some normative ethical principals applicable to pedagogic research and to formalize the ethical review of pedagogic research through a process of ethical review.
Institutional Review Boards and the Process of Ethical Review
Institutional Review Boards (IRBs) exist to minimize or eliminate avoidable harm to humans who are the subjects of research. These boards require that researchers consider the potential harms to which they may expose their subjects and make every effort to eliminate those harms. Risks are to be minimized and any remaining risks must be shown to be reasonable in relation to their potential benefits. Informed consent is required to respect and protect the autonomy of those involved.
While these regulations, and the training that accompanies them within most institutions, do identify some of the basic ethical principles of human subject research: respect for persons, beneficence, and justice they also delineate the distinction between practice and research, and further identify the types of research subject to IRB review. This latter distinction is especially important in the context of pedagogic research. While recognizing the need to distinguish between formal research and variations in standard clinical practice and while identifying what is “experimental” in both, only formal research demands IRB review. Further governmental agencies do not include much of pedagogic research with other forms of research that are subject to IRB review. Why not? Informal experimentation/research in the classroom regarding methodologies and curricula appears to be exempt from federally mandated IRB review. Indeed, as the Code of Federal Regulations, Title 45, Part 46, 2001 states that “research conducted in established or commonly accepted educational settings, involving normal educational practices…” is not required to submit to IRB review.
This type of exempted (informal) pedagogical experimentation and research includes evaluation of instructional strategies, instructional techniques, educational testing, and even the collection of existing data (provided that subject confidentiality is maintained). Yet if the basic ethical principles underlying the need for and operation of Institutional Review Board supervision are to be taken seriously they would indicate a need to have them considered in the design and conduct of any pedagogic research whether formal or informal, whether major studies or small changes in clinical practice. The exemptions or waivers form the submission of research proposals to an IRB for informal educational experimentation/research does not absolve researchers from preceding ethically, a basic fact rightly recognized by Schulman and Hutchings. (Shulman, 2003, Hutchings, 1993)
Currently, the only ethical review of pedagogical research proposals is conducted through IRBs, and there is a need on the part of the profession of post-secondary education to establish the need for any and all pedagogic research to be submitted to an informal ethical review process, one which is not “institutional” per se, but which derives from the professional responsibilities of all educators as fiduciaries to cause benefit and to minimize avoidable harms.
We believe that the lack of IRB review makes the awareness of the ethical issues inherent in classroom experimentation even more important. As the rapidly growing literature on action/practitioner research and pedagogic ethics shows, pedagogic ethics has become a central element in the debates concerning teaching effectiveness and informal pedagogical assessment.
An ethics of pedagogic research demands we pay attention to the underlying ethical principles of the professional obligations of faculty members as both teachers and researchers. And in this sense, it is the construction of a context-sensitive yet normatively significant framework of ethical guidelines that becomes the crucial next step in the growing field of the scholarship of teaching and learning. Normative discourse is possible once post-secondary educators realize that they voluntarily assumed a dual set of responsibilities when they entered the profession of education and decided to perform research in both their respective disciplines and with regards to their teaching. Therefore, all post-secondary educators have both the fiduciary responsibilities towards their learners to benefit them, and the basic responsibility of a human subject researcher to cause no harm. Within this framework of dual responsibilities, we can better understand the ethics of pedagogic experimentation.
Students as a Vulnerable Category and the Potential for Harm
The concept of harm in pedagogy has received little attention, yet it is one of the most significant issues relating to the ethics of education. There are a wide variety of potential ways educators can and do cause significant harm to their students. Because student success is partially dependent on decisions faculty members make regarding instructional design, course assignments, assessments and grading (among others) there is always the risk that their academic achievements and success could be negatively impacted by these decisions. This is the case whether or not a pedagogical experiment is being conducted, which is why we refer to the teacher/student relationship as fiduciary. The entire teaching profession operates on the assumption that teachers will make decisions of the sort described above that are in the best interests of their students. When pedagogic research is involved, the stakes are even higher because the research may be intended, in part, to further the career of the teacher/researcher by generating publications or conference presentations.
Yet even if the research or experimentation does not fall under the auspices of IRB review, the ethical dilemmas remain. Suppose there is a change in a pedagogic technique (research-experiment) or a new instructional design utilized by an educator, these changes could result in some learners not doing as well as they may have had the change not been made. Not doing as well as one could have constitutes a harm to the learner and it can be manifested in a variety of ways.
Suppose a faculty member has been offering instruction with a common instructional design for class that incorporates the basic lecture format with a traditional short answer objective items form of assessment. Now the faculty member (with encouragement from the university administration) wants to introduce a change to determine what the difference might be in terms of learning objective outcomes. The change involves having the students make individual presentations in class and a greater emphasis on working in small groups. Some of the members of class have problems working in groups, others have performance anxieties and find it very difficult to speak before others, even in the smallest groups. These learners do very poorly on the new assignments and as a result receive poor grades and their final grades are less than they generally get in classes using the more common lecture format. These learners have thus suffered a lower grade, a lowering of their GPA. Some withdrew from the class rather than needing to experience speaking before a group. Those that withdrew will need to take the class at another time with another instructor and they have hade their progress to graduation impeded. So something so simple as having students speak in class or work in small groups can produce ill effects.
Because the subject of the potential harms to students is not explored extensively in the existing literature on the ethics of pedagogic research we offer here a partial list of potential harms:
• --a decline in completion and retention rates
• --a decline in success rates/GPA
• --the inability to perform at the next level of study
• --the inability to develop and use skills that are needed beyond the classroom
• --a failure to develop critical thinking skills
• --a failure to develop information processing skills
• --a failure to acquire new knowledge
• -- the inability to function as a fully educated member of a democratic society
• --the inability to realize socio-economic goals (career, etc…)
• --a decrease in self esteem
• --an increase in hostility towards the educator/institution
• --a negative impact on future educational success (self-efficacy)
• --loss of time
• --loss of tuition
• --student loans
Pedagogical Ethics: A Guide for Practical Action
Given these potential harms to learners, what is required of professional educators regarding their educational practice and pedagogical experimentation? First, it means that each and every educator “come to terms” with inherently ethical dimensions of their practice, and that each one be willing to engage in the difficult process of ethical evaluation and decision-making.
Second, this process of decision-making and evaluation need not take place in a vacuum. And while what follows is far from a comprehensive listing of the decision-making process and principles at work in pedagogical ethics, it can be seen as a rudimentary guide by which to navigate the processes of ethical review.
A Design for Ethical Review of Pedagogic Research Involving Human Subjects
In any pedagogical experiment, the first step is to determine whether or not the experiment is likely to cause harm to the human subjects involved. There are several things an educator can do to reach a determination as to the possibility or degree of potential harm. The most direct is to rethink the proposed experiment from the perspective of the student and consider the possible impact of the experiment or the changes in pedagogic method or materials on the learner and the learning. If there appear to be significant risks of impeding the learning process for some students or the risk of discouragement or the inducement to withdraw from the learning or the class itself then the project would need to be seriously reconsidered if not abandoned altogether if the risks were high and the probability of harm was high. This would be the case regardless of the potential value of what might be learned. If the risks were not high and the probability of harm was low then they would be weighed against the potential value of what might be learned.
Another indispensable and invaluable part of the process of determining risk is to conduct a search of the literature to determine if the experiment has already been performed and if so, what were the outcomes. This sort of thing is common place in medical research, a field that should provide several paradigms for pedagogic research. This literature search would include a search for similar experiments with similar groups of learners (subjects). If it has already been done and there were harmful outcomes then the repetition of the experiment would be professionally suspect. If it has already been done and there were little or no reported harmful outcomes then a repetition of the experiment to confirm the findings would be ethically condoned. If it has already been done and there were little or no reported harmful outcomes, then the educator proposing the project would need to reexamine the need for doing the research and instead consider adopting the pedagogic innovation previously tested and proven to be both effective and safe.
The second step in preparing for a pedagogic experiment with human subjects is to determine whether or not the experiment is subject to institutional review. Also important are the policies and procedures unique to each institution. And if the experiment does indeed fall under the purview of IRB review, the next step lies in contacting the board itself. They will then conduct a thorough review of the proposed experiment.
However, the real work of pedagogical ethics begins with the review of the proposal for potential harms even if an experiment in the classroom does not need IRB approval. For too long, educators have “quarantined” the ethics of education in the IRB office. Yet the ethics of pedagogical experimentation involves some tough questions beyond the legalistic concerns of institutions. Am I causing harm to any of my students? How can I minimize the risk of harming my students? Do the potential pedagogical benefits justify the potential risks to my students? Does the proposed experiment maintain or strengthen my professional obligations and commitments?
On these grounds, another important part of the ethical review process (ERP) involves investigation and communication with colleagues. There is a huge growth in literature on pedagogy, and researching the topic is relatively painless, given the advances in internet technology. With colleagues, the question becomes “have you tried this?” Colleagues owe it to their fellow teachers/researchers to make known the results, whether successful or otherwise, of their experiments, large and small, formal and informal, funded and unfunded. When communicating and disseminating the results of this type of pedagogical experimentation, college and university professors begin to fulfill their professional obligations as educators. With the results widely disseminated, educators can have more fruitful searches of the literature to better determine both the safety and efficacy of the pedagogic techniques and materials they were thinking of using in their own experiments.
A Design for the Ethical Review of Informal Pedagogic Research Involving Human Subjects
As previously mentioned, the IRB has very explicit guidelines in the principal investigator (PI) manuals they provide to researchers regarding certain issues pertaining to the protection of human subjects. Those guidelines that are most usually and readily applied to pedagogic research include the necessity of informed consent and associated specifics about what goes on the consent form. These details include the reading level/comprehensibility of the language used and the administration of informed consent including who, when and how it is administered. The IRB requirements for human subjects research are also very specific about the need to avoid coercion when soliciting student participation and about the need to insure confidentiality and privacy through the proper handling of any and all student information that is collected in conjunction with pedagogic research.
It is our contention that other aspects of this type of research can and should be more carefully constructed and reviewed in order to avoid potential harms to students, like those listed above. To this end we propose an Ethical Review Process (ERP) comprised specifically to heighten awareness of fundamental ethical issues and to assist pedagogic researchers in the design of projects that will protect students (as human subjects) from the kinds of harms that are particular to the educational environment.
What follows is a list of potential principles and guidelines for the Ethical Review Process we propose:
1. The ERP should precede (and coincide, where applicable) review by the IRB, as the ethical review process should help to refine the research proposal and ultimately expedite the proposals smooth passage through the IRB.
2. The ERP is not intended to function as a barrier to pedagogic research, but is rather intended to facilitate and support research that will provide the greatest benefits to students and to the profession while avoiding potential harms.
3. The ERP is not intended to substitute for IRBs and all research that is required by law to be submitted to the IRB should pass through that process regardless of ERP review. Equally, the ERP is not intended to protect colleges and universities from litigation, but solely to emphasize the ethical dimensions of pedagogic research in a given community, to provide knowledge-rich support for potential faculty researchers and most of all to provide an additional layer of protection for students..
4. The ERP should include people with expertise in educational theory and practice, pedagogy and pedagogic research. Their work should be understood by them and by the college or university as a crucial contribution to a community of pedagogic practice and inquiry.
5. In the context of the scholarship of teaching and learning, the ERP should be understood as a mechanism that enhances the ethical responsibilities of post-secondary educators by calling on the inherent obligations and commitments that come with the professional role of faculty members; namely, to seek knowledge, to share what our investigations uncover and to contribute to the larger community of scholars and practitioners.
6. Pedagogic research projects submitted to the ERP should include a literature review that demonstrates that the researchers have familiarized themselves with the existing literature related to the proposed research in an effort to avoid subjecting students to potential harms.
Ethical Considerations and Decision-making in Pedagogic Experimentation
We presented a range of ethical issues associated with pedagogical experimentation that included the potential for harm to learners, considerations of experimental design, informed consent, privacy, and the professional obligations of the educator as both researcher and teacher. Shulman, Hutchings, and others have demonstrated, pedagogical ethics is not so much concerned with coming up with “right” answers, but in asking the right sort of questions. We identify, within a framework of duties and obligations, some of the questions that educators should be asking themselves when they experiment in the classroom.
1. Am I causing harm to my students?
2. Am I providing benefit?
3. Am I fulfilling my professional obligations as a teacher and a researcher?
The first of these questions relates to the duties and obligations generated by being a human being and the very least of these demands that we treat others with respect and dignity, and refrain from harming others, regardless of our “status” in the relationship. Primum non nocere: First, Do No Harm
Second, educators qua educators (as members of the profession) are also bound by duties of beneficence. As fiduciaries, educators are bound to act in the best interests of their students, to promote effective learning, and to fulfill the requirements of educational institutions regarding democratic citizenship.
Finally, as researchers and experimenters, post-secondary educators have (in addition to the obligations of beneficence and non-malfeasance) the obligation to maintain a commitment to responsible inquiry, and to the advancement of the profession of education itself.
Many discussions of ethics with educators center around legal and practical concerns, and very few evidence substantive reference to fundamental ethical principles, thus undermining the very real concern educators have for the learner/subject. The limiting of concern to the legal requirements being met are evidence of practitioners operating out of the “codal” model of the relationship of educator to learner and researcher to subject. This model implies “here are the rules, don’t break them.” Yet this interpretation of ethical principles takes on a defensive posture, one generated more for concern of the members of the profession than for concern over the well being of those served by the profession: the learner/subject. It is this defensive approach to the ethics of research into the effectiveness of post-secondary education that the ERP is intended to ameliorate.
Among the professional responsibilities of educators is the responsibility to conduct pedagogic research and experimentation in order to improve on the efficacy of instruction. This responsibility together with the fiduciary responsibilities to cause benefit and avoid harm to learners leads to several essential ethical concerns with pedagogic research with human subjects related to safeguarding them from harm. While there are criteria of exemption from the IRB process there is not any such exemption from the responsibility of every educator-researcher-experimenter for an ethical review of every pedagogic experiment, innovation or change. In this essay, we have tried to lay the foundations for the process of ethical review in pedagogical innovation by emphasizing the basic professional obligations of all post-secondary educators.
Burgess, R. G. (1989) The Ethics of Educational Research. London: Falmer Press.
Cohen, Patricia ( 2007) “ As Ethics Panels Expand Grip, No Field Is Off Limit” New York Times, February 28, 2007.
Hutchings, Patricia A. (1993) Using Cases to Improve College Teaching: A Guide to More Reflective Practice. Washington, D.C.: American Association of Higher Education.
Hutchings, Patricia, A. (2003) “Competing Goods: Ethical Issues in the Scholarship of Teaching and Learning.” Change. September/October, 2003. pp. 27-33.
Hutchings, Patricia A. (2002) (ed) Ethics of Inquiry: Issues in Scholarship of Teaching and Learning. Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching .
Kincaid, Shannon and Pecorino, Philip (2006) “Ethical Issues in Pedagogical Research,” Community College Humanities Review, Vol. 26, No.1, Fall.
Pecorino, Philip and Kincaid, Shannon (2007) “Why Should I Care About SOTL?" International Journal for the Scholarship of Teaching and Learning, Vol. 1, No.1, Jan. 2007.
Shulman, Lee S. (1988) “Disciplines of Inquiry in Education: An overview.” In Richard M. Jaeger (Ed.), Complementary Methods for Research in Education (pp.3-20). Washington, DC: American Educational Research Association
Shulman, Lee S. (1993) “Teaching as Community Property: Putting an End to Pedagogical Solitude.” Change, 25(6), pp. 6–7
Shulman, Lee S.(1999) “Taking Learning Seriously.” Change. Vol. 31, No. 4, pp. 10-17.
Shulman, Lee S. (2000) “From Minsk to Pinsk: Why a Scholarship of Teaching and Learning?” Journal of Scholarship of Teaching and Learning (JOSOTL), 2000, 1 (1), pp.48-51 http://titans.iusb.edu/josotl/VOL_1/NO_1/SHULMAN.PDF
Shulman, Lee S.(2002) Preface in Hutchings, Patricia A. (ed) Ethics of Inquiry: Issues in Scholarship of Teaching and Learning. Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching,
Philip Pecorino teaches philosophy and ethics at Queensborough Community College of the City University of New York. If you have questions or comments about this article, please contact him at firstname.lastname@example.org.