In the late 1800s, the prevailing myth held that men were more intelligent than women. Mary Calkins, a psychologist, conducted experiments at Wellesley College in 1887 that demonstrated that women are just as intelligent as men.
—Furumoto, 1980
Compelling pedagogical interests require that each program prepare graduates to navigate cultural and individual differences in research and practice, including those that may produce value conflicts or other tensions arising from the intersection of different areas of diversity.
—American Psychological Association Commission on Accreditation, 2016, p. 8
What clinical experiences have enhanced completer’s understanding of diversity and equity issues and their readiness to use that understanding in teaching situations? What applications of technology have prepared completers for their responsibilities on the job?
—Council for the Accreditation of Educator Preparation, 2018, p. 37
How can school counselors help students in low-income schools get good grades?
—Williams, Steen, Albert, Dely, Jacobs, Nagel, and Irick, 2018, p. 156
The ways of Indigenous research are as old as the hills and the valleys, the mountains and the seas, and the desert and the lakes that Indigenous people bind themselves to as their places of belonging. It is not that Indigenous peoples are anti-research … the “bad name” that research has within Indigenous communities is not about the notion of research itself; rather it is about how that research has been practiced, by whom, and for what purpose that has created ill-feeling.
—Cram, Chilisa, and Mertens, 2013, p. 11
Why Bother?
Life is complex; the world is not perfect. Many different kinds of people live on this planet, and educators and psychologists do not know the best ways to educate or counsel many people who have a history of poor achievement in school and who suffer a poor quality of life in terms of illiteracy, physical and mental illness, low pay, poor working conditions, high rates of unemployment, and other social and psychological disadvantages. The brief descriptions presented at the beginning of this chapter illustrate the importance of attending to all learners and clients with respect to cultural responsiveness and the complexity of educational and psychological challenges that confront researchers in our society. They highlight the importance that accreditation organizations place on developing research skills that equip educators and psychologists to address these challenges. They also give us pause to think about the role that research can play in providing insights into how research can contribute to changing the life experiences of those who suffer discrimination and oppression.
What Is Research?
Research is one of many different ways of knowing or understanding. It is different from other ways of knowing, such as insight, divine inspiration, and acceptance of authoritative dictates, in that it is a process of systematic inquiry that is designed to collect, analyze, interpret, and use data. Research is conducted for a variety of reasons, including to understand, describe, predict, or control an educational or psychological phenomenon or to empower individuals in such contexts.
The exact nature of the definition of research is influenced by the researcher’s theoretical framework and by the importance that the researcher places on distinguishing research from other activities or different types of research from each other. For example, many students go to the Internet or the library and look up facts from a variety of sources and say that they are doing a research paper. Some journalists follow a similar search strategy and often include interviews with people close to the action that is the focus of a news report. The focus of this text is NOT on that type of “research.” Rather, this text focuses on empirical research that is characterized as building on existing knowledge about a phenomenon. This base of knowledge (whether derived from scholarly literature or community interaction) is used to develop a research focus and questions and/or hypotheses as well as systematic collection of data from selected participants. The data are analyzed, interpreted, and reported. Such empirical research is found in scholarly journals, although this is not the only source where empirical research can be found.
Extending Your Thinking Definition of Research
One definition of research is provided in this text. Think about your own understanding of what it means to do research. Explore other definitions of research in other texts or through the Internet. Modify the definition provided or create a new definition that reflects your understanding of the meaning of the term research.
Two parallel genres of inquiry in the educational and psychological communities have grown side by side: research and program evaluation. At times, these two genres intersect; at other times, they follow very separate trajectories. The relationship between research and evaluation is not simplistic. Much of evaluation can look remarkably like research and vice versa. Both make use of systematic inquiry methods to collect, analyze, interpret, and use data to understand, describe, predict, control, or empower. Evaluation is more typically associated with the need for information for decision-making in a specific setting, and research is more typically associated with generating new knowledge that can be transferred to other settings. In practice, a large area of overlap exists between evaluation and research. Hence, what students learn in their study of research has application in their understanding of evaluation as well. The contextual factors and approaches unique to evaluation are described in the next chapter so that readers who are interested in evaluation can use the methodological guidance in subsequent chapters to plan an evaluation study.
Research Terminology
Like most disciplines, researchers have their own jargon that has meanings different from everyday uses of the same terms. If you have studied research before, you might be familiar with these terms. However, it is almost impossible to talk about research without having at least a rudimentary understanding of these terms. Therefore, if you are new to the researcher’s world, you should stop and review the terms and definitions presented in Box 1.1. Make sure you mark this box in your textbook so you can refer to these definitions because these terms appear often in the following chapters.
Box 1.1 Research Terminology: Definitions and Examples
Quantitative/qualitative/mixed methods: The description of these methods is the heart of this entire text. In quite simplistic terms, quantitative researchers collect numerical data; qualitative researchers collect words, pictures, and artifacts. Mixed methods researchers collect both types of data.
Subject or participant or stakeholder: The individual you are studying is the subject or participant; this is the person from whom you collect data. The term subject was used more frequently in the past and can still be seen in some journals. More recently, the term participant is used in recognition of the active role that human beings play in the research process as contributing participants. Hence, this is the term that is generally used in this text. Often, the participant in educational and psychological research is a student, client, teacher, administrator, or psychologist, but it could also be an animal or a textbook. For example, in Christodoulou et al.’s (2017) study of the effects of a summer reading program, the participants were 47 students, aged 6 through 9, who had been diagnosed with a learning disability. NOTE: Stakeholder is a term that is sometimes used (more frequently in program evaluation) to indicate members of the community who have a “stake in the outcomes of the research.” Stakeholder is usually more inclusive than the terms subject or participant because it can include those from whom data are collected as well as administrators, staff, and others in the community who will be affected by the results of the inquiry.
Independent variable and predictor variable: The independent and predictor variables are the variables on which the groups in your research study differ, either because you have exposed them to different treatments (independent variable) or because of some inherent characteristics of the groups (predictor variable). When the researcher deliberately manipulates a treatment (e.g., introduces literacy training for one group but not the other), the treatment is called the independent variable. Common independent variables in education and psychology include variations in methods of teaching or therapy. Christodoulou et al. (2017) had an independent variable that was the Seeing Stars reading program. If the researcher is interested in the effect of differences of an inherent characteristic, the variable is more frequently called a predictor variable. For example, in studies of gender differences, gender is the predictor variable.
Dependent variable and criterion variable: The dependent or criterion variable is the variable that the researcher is interested in measuring to determine how it is different for groups with different experiences (dependent) or characteristics (criterion). The dependent variable gets its name because it depends on what the researcher does with the independent variable. The researcher manipulates an independent variable (treatment) and exposes groups to differing amounts or types of it and then measures a dependent variable to see if it is different for the different groups. For example, in the Christodoulou et al. (2017) study, one dependent variable was timed-reading ability as measured by the Test of Word Reading Efficiency–2. When working with a predictor variable (inherent characteristic or nonmanipulated variable), the measurement of “effect” is called a criterion variable. Common dependent or criterion variables in education and psychology include academic achievement, social skills, personality measures, and income after leaving school.
Experimental and control groups: In certain types of research, the researcher can divide the participants into two or more groups to test the effect of a specific treatment (independent variable). For example, a researcher might want to test the effect of providing social skills training to students with disabilities by comparing outcomes for students who receive such training with those who do not. The group that receives the training is called the experimental group. The comparison group that does not receive the training is called the control group. In some research studies, participants are randomly assigned to conditions—that is, they have an equal and independent chance of being assigned to either the experimental or the control group. Christodoulou and colleagues (2017) studied the effect of a summer reading program for students with a learning disability. Based on random assignment, the students who participated in the summer reading program were the experimental group; the students placed on a wait list were the control group. A researcher can also study the effect of a treatment without manipulating it or comparing groups who do and do not receive it. This is commonly done in qualitative and descriptive research studies in which researchers and evaluators theorize the conditions/interventions necessary for change to occur and then collect data to determine the extent to which that change did occur (Gates & Dyson, 2017).
Population and sample: The population is the group to whom you want to apply your results. The sample is the group that you have chosen from your population from which to collect data. For example, researchers might have access to 3,000 students. Rather than collect data from all 3,000 students, they might choose 300 students to include in their study (10% sample).
Generalizability and transferability: Generalizability refers to the researcher’s ability to generalize the results from the sample to the population from which it was drawn. The ability to generalize results depends on how representative the sample is of the population. The degree of generalizability can be discussed in statistical terms, depending on the type of sampling strategy that the researcher uses. For example, the researchers who select the 300 students might want to generalize their results to the 3,000 students in the population. In qualitative research, the researcher emphasizes the total context in which the research takes place to enable readers to make judgments as to the transferability of the study’s results to their own situations.
Statistically significant: Statistical significance is important in studies in which comparisons between groups or estimations of sizes of relationships between variables are made. If groups are compared on a dependent variable (e.g., social adjustment or literacy skills), a test of statistical significance can be used to determine if the observed difference between the groups is too large to occur plausibly as a result of chance alone. On the basis of the laws of probability, a difference that is too large to attribute to chance is called statistically significant. Researchers in education and psychology will sometimes say that their results are statistically significant at the 0.05 or 0.01 level. These levels refer to the researchers’ confidence that similar results would probably be obtained if the study were repeated using other samples drawn from the same population.
Extraneous/lurking variables (also known as moderating or intervening variables): Researchers are typically very interested in the effect of their independent (or predictor) variables on the dependent (or criterion) variables. But social phenomena are complex and are influenced by many variables other than those of central interest to the researchers. These other variables that can influence the effect of the independent or predictor variables are called extraneous variables. For example, a researcher might be very interested in testing the effectiveness of a new therapeutic or teaching approach. However, the participants might have varying degrees of enthusiasm for the different treatments. The counselors or teachers might be strongly wedded to the traditional approach, or they might be intrigued by the new ideas represented in your experimental treatment. Thus, it may be the extraneous variable of their enthusiasm that determines which approach produces the more desirable outcome rather than the approach itself. Other common extraneous variables can be associated with culture, gender, disability, ability, and ethnicity differences between groups.
Community-based participatory research: Community-based participatory research typically means that the research involves community members and researchers in a partnership in which they serve as members of a research team, contributing to the design and management of the research. The goal is to work toward community change through cycles of action and reflection (Mullett, 2015). There are different labels that are used to describe this approach to research, including participatory action research, cooperative or collaborative research, or simply action research.
Extending Your Thinking Research Terminology
For each concept listed in Box 1.1, provide a definition in your own words and an example from a research study.
Approach Taken in This Book
The main focus of this text is to examine, from a variety of philosophical and theoretical perspectives, the process of systematic inquiry that constitutes research and evaluation in education and psychology. The typical process for planning and conducting a research or evaluation study is displayed in Box 1.2. This process is rarely as linear as this figure suggests; it can be very iterative in nature. Although these steps are used to organize the information in this text, in actual practice, the researcher may take one step forward, three steps back, and then jump to Step 4, only to find it necessary to revisit Step 2.
In fact, the nonlinearity of planning and conducting research suggests that readers may choose to use this book in a nonlinear fashion. The first three chapters do provide an overview of the nature of research and evaluation and how to begin identifying a research topic. It would seem prudent, therefore, to begin with those chapters (although readers may choose to skip the chapter on evaluation if that is not included in their course syllabus). If readers have a goal of designing a research proposal, they might start in the appendix to read about how to develop a research proposal and use that as a guide to deciding how to navigate through the rest of the text.
After that, readers might choose to read any of the subsequent chapters on specific research approaches (e.g., experimental design) and then complete their understanding of the process for that approach by reading the last three chapters on sampling, data collection and analysis, and reporting. Readers could then return to earlier chapters to learn about other approaches to research and build on what they learned in the first go-round with the text. Alternatively, readers who have a strong feeling that a specific research strategy is of interest to them could start with the chapter on that approach (e.g., survey research) and then jump to the last three chapters of the book.
Some research methods textbooks address quantitative research methods (research that measures variables in a quantifiable way) or qualitative research methods (research that captures holistic pictures using words). (These definitions are overly simplistic; they are expanded in later chapters.) An increasing number of books and journals have begun to focus on mixed methods research. In this book, I make the assumption that readers need to understand both quantitative and qualitative approaches to research before they move to mixed methods. Several of the sample studies used throughout the text use mixed methods and there is a separate chapter focused specifically on this approach.