Preview Extract
Introduction to Quantitative Research CHAPTER OVERVIEW Chapter 9 provides an overview of the meaning, purpose, and importance of quantitative research design. The main purpose of the research design is identified as providing a scheme for answering specific research questions. To prevent bias from impinging on the dependent variables and thereby affecting study outcomes, the following considerations are presented: Objectivity in the conceptualization of the problem Accuracy in all aspects of the study Feasibility of obtaining subjects, allocating time and financial resources, and analyzing data Control over tested and extraneous variables Possible ways in which control can be accomplished include homogeneity or similarity with respect to extraneous variables and sampling, constancy in data-collection conditions and procedures, manipulation of independent variables, and randomization in sampling techniques and data- collection instruments. Internal and external validity are defined in this chapter. Threats to internal validity are identified as the effect of history on the dependent variables, the maturation processes operating within study subjects, the effect of pretest taking on post-test results, inconsistency of instrumentation, mortality (dropout) of study subjects, and biased selection of the study sample. Threats to external validity include effects of selection, reactive effects, and effects of testing. In critiquing the research design, students should be aware of the overall implications that the choice of a particular design may have for the study as a whole and the means used to maximize control. LEARNING OUTCOMES After reading this chapter, the student should be able to do the following: Define research design. Identify the purpose of the research design. Define control as it affects the research design. Compare and contrast the elements that affect control. Begin to evaluate what degree of control should be exercised in the design. Define internal validity. Identify the threats to internal validity. Define external validity. Identify the conditions that affect external validity. Evaluate the design using the critiquing questions.
CHAPTER 9 Introduction to Quantitative Research 45
1. TEACHING STRATEGIES As consumers of research, nurses must be able to critically evaluate the design of a study in terms of its validity. Therefore the following teaching strategies are designed to facilitate students’ learning of quantitative research design fundamentals, especially those related to control, so that they can achieve this goal. 1. Critical Thinking Challenges Based on the content of Chapter 9, the following questions are suggested to review important concepts and stimulate critical thinking: RECALL AND UNDERSTANDING What is random sampling? Random assignment? “A researcher attempts to use a design to maximize the degree of control over the tested variables. Control involves holding the conditions of the study constant” (Chapter 9 in the textbook). What are four ways in which a researcher can maximize control in a study? What are common threats to the internal validity of a study? What conditions affect the external validity of a study? ANALYSIS AND SYNTHESIS What are the similarities and differences between random sampling and random assignment? How does a researcher accomplish the four types of control referred to in “Recall and understanding” above? How do internal and external validity relate to each other? 2. Differentiating Concepts: Random Sampling and Random Assignment Students often have diffi culty understanding the difference between random sampling and random assignment. It is important to describe these two processes clearly and then to c ompare and contrast them several times throughout the course. Emphasis should be placed on the r esearcher’s ability to randomly assign subjects to experimental and control groups, even if the sample was not initially obtained in a random manner. The following comparison highlights the differences between these two methods of randomization. RANDOM SAMPLING Deals with selection of a study sample from a larger population May be used in any research design RANDOM ASSIGNMENT Deals with the manner in which subjects, already selected from a larger population, are put into treatment groups Usually reserved for experimental or quasiexperimental design Examples of both random sampling and random assignment should be presented. Here are some suggestions: Example: random sampling A nurse researcher is interested in studying the relationship between independence in activities of daily living and self-concept in a population of elderly nursing home residents. To conduct her study, the researcher has gained access to the residents of three nursing homes that are of comparable size and have similar facilities. Each institution has provided her with an alphabetical list of names of their residents. The total population is 700. For this particular study, a sample of 150 is adequate. To determine which residents to include in the study, the researcher gives each of the 700 residents a number and then uses a table of random numbers to select 150 of the residents for inclusion in her sample. Example: random assignment A clinical nurse specialist in surgery and a faculty member from a nearby school of nursing have decided to study the effects of therapeutic touch on postoperative pain. After gaining approval for their study from the institutional review board of a large urban medical centre, they begin data collection. Because the total population of postoperative patients who will meet the inclusion criteria is not known in advance, they decide to give each potential subject a number on the basis of chronological admission to the study. For example, the fi rst patient to consent to participate in the study is assigned No. 1, the second patient No. 2, and so on. The sample for study will consist of 60 patients. The study has a control group (patients who receive the standard preoperative care provided at the institution) and an experimental group (patients who receive standard care and therapeutic touch). Using a table of random numbers, before the beginning of data collection, the researchers assign 30 numbers to each of the two groups. As patients are admitted to the study, their subject number determines the group in which they will be placed. For example, the fi rst, fi fth, and seventh subjects are to be assigned to the experimental treatment, and the second, third, fourth, and sixth subjects assigned to the control group. Although a purposive, nonrandom approach has been used for sampling in this study, subjects will be randomly assigned to experimental and control groups. 3. Providing Experiential Learning Activities a. Identifying Threats to Internal Validity The purpose of this exercise is to allow students to practise identifying threats to the internal validity of studies. Student objectives for the learning activity: • Determine the threats to internal validity in each selected study. Directions for implementation. After students have read Chapter 9 in the textbook and engaged in a discussion about internal validity in class, give them a copy of Figure 9-1 found at the end of this chapter. Ask students to read the summaries of the studies and identify the threats to internal validity posed in each one. You may decide to use all of the studies presented or selected ones, depending on the amount of time you wish to allot to this exercise. Reading the studies and identifying and discussing the threats should take approximately 1 hour. To conserve class time and give students an opportunity to review the concepts previously presented, the exercise should be completed after class and discussed in a subsequent class. Because this exercise is meant to be a learning experience and not a method of student evaluation, it is important that the students’ papers not be collected. Students should use this experience for selfevaluation of their knowledge of this content area. Answer key: S: Because women were not randomly selected for inclusion in the study, other factors, not controlled, may provide an alternative explanation for the study findings. I: Baseline and post-treatment blood pressures were obtained by different individuals using different instruments. E: Subjects in the experimental group dropped out of the study; those in the control group did not. S: Subjects were not randomly assigned to treatment groups. Other factors may account for their behaviour change.
CHAPTER 9 Introduction to Quantitative Research 47
T: Subjects were exposed many times to the same test. This may have influenced their responses. H: Events external to the study treatment, such as the employment of a new head nurse on the experimental unit, may offer an alternative explanation for study findings. M: Because the children were not compared with a control group who did not receive this treatment and no randomization was employed in the study design, it is possible that the usual process of growth and development could account to some extent for the study findings. 4. Applying Concepts and Principles: Relating the Research Design to the Purpose of a Study If students are to gain the ability to determine the appropriateness of research designs used in various types of studies, they must be provided with an opportunity to practise this skill. One way to accomplish this is to present students with several examples, actual or hypothetical, of the problem statement or the purpose of a study. Then ask them to identify the type of research or research design that would be most appropriate for s olving the problem or achieving the purpose of the study. This strategy may be used either to review previously learned content or to provide the initial impetus for discussion of new content. Three examples of study purposes or problem statements are provided below, along with an indication of the appropriate research design/type. P1: W hat is the effect of problem-oriented counselling and peer group pressure on smoking cessation in adolescents? (Experimental) P2: The purpose of this study was to assess the reliability and validity of the Pain Beliefs Questionnaire, an instrument to measure the degree of personal control that individuals perceive they have in affecting their pain experience. (Methodological) P3: The purpose of this study was to compare the effects of two cleansers on wound care and healing over a 3-week period. (Quasiexperimental, time series design)
FIGURE 9-1 Exercises on Internal Validity (Modified from While, K. (1984). Exercises on internal validity, June 12–13 (pp. 81–84). Unpublished manuscript. Workshop for the Amy Research Institute, Washington, D.C.) Experimental and Quasiexperimental Designs CHAPTER OVERVIEW Chapter 10 presents a refinement of the previous chapter’s content on research design; it focuses on the issues, features, and evaluation of specific types of experimental designs. A case is made for the determination of cause-and-effect relationships through the use of experimental designs because they help eliminate alternative explanations for the findings of a study. Criteria for inferring causality are stated. Also, the point is made that experimental designs do not dominate nursing research studies because of issues in the development of nursing theory (as defined in Chapter 4) and the paucity of well-defined variables that are relevant to developing nursing knowledge. The properties of a true experimental design—randomization, control, and manipulation—are said to strengthen the ability to infer cause-and-effect relationships. Two other types of experimental design are identified: the Solomon four-group design and the after-only experimental design. In addition, field and laboratory experiments are compared. Advantages and disadvantages of experimental designs are discussed, including inherent validity problems. Quasiexperimental designs are contrasted with true experimental designs, and the ones most commonly used in nursing research are discussed in detail. Again, advantages and disadvantages of these designs are presented. The section on evaluation research discusses the purposes and uses of systematically studying program outcomes. A step-by-step “how to” critique of different designs is offered. This provides students with a means to approach a somewhat complex task.
CHAPTER 10 Experimental and Quasiexperimental Designs LEARNING OBJECTIVES After reading this chapter, the student should be able to do the following: List the criteria necessary for inferring cause-and-effect relationships. Distinguish the differences between experimental and quasiexperimental designs. Define internal validity problems associated with experimental and quasiexperimental designs. Describe the use of experimental and quasiexperimental designs for evaluation research. Critically evaluate the findings of selected studies that test cause-and-effect relationships.
CHAPTER 10 Experimental and Quasiexperimental Designs 51
TEACHING STRATEGIES Because Chapter 10 in the textbook builds on the material presented in Chapter 9, by describing the way in which specifi c designs manage issues of control and threats to validity, the teaching s trategies presented here build on those discussed in Chapter 9 of the Instructor’s Manual. It is our intention to not only give suggestions for teaching new content, but also reinforce what the student has learned about general design issues, specifi cally threats to validity. 1. Critical Thinking Challenges The following questions can facilitate student comprehension and critical thinking about this chapter’s content: RECALL AND UNDERSTANDING What is the major difference between experimental and quasiexperimental designs? How would you describe the three properties of true experiments? What are extraneous variables? What factors in clinical research may preclude the use of true experimental designs? Can you identify and describe three types of quasiexperimental designs? What are the purposes and uses of evaluation research? ANALYSIS AND SYNTHESIS Several laboratory studies were conducted using true experimental designs. They all supported the effi cacy of a new method for administering subcutaneous heparin injections. What would you consider important when thinking about readiness for implementation to practice? (Note: The emphasis here should be on helping students to think through the need for including clinical studies in a research base before utilization.) How can nurses use evaluation research to enhance clinical practice? Using Examples of Actual Hypothetical Studies to Illustrate Abstract Designs Several specifi c designs that are categorized as experimental or quasiexperimental are present in Chapter 10. The concepts and principles related to these designs are abstract. As stated previously, students often have diffi culty understanding abstract concepts and principles that are not within their realm of experience. Therefore a helpful strategy is to use numerous concrete examples of the concepts introduced to students. You may use summaries of actual or hypothetical studies to illustrate each type of design presented or discussed in class. Involving Guest Speakers Another approach used to illuminate research design, and one that students fi nd interesting, is to have a faculty member or graduate student from our school present his or her experimental or quasiexperimental study to the class. This not only demonstrates the practical application of design principles, but also exposes students to the reality of nursing research. It is helpful to provide the guest speaker with guidelines for the presentation, so that the focus for this class will be on the research design. More specifi cally, reasons for the choice of design, how the design controls for extraneous variables, limitations of the design, and alternative explanations for study fi ndings are appropriate areas to emphasize. The presentation should take no more than 30 to 45 minutes. An additional 30 minutes should be spent discussing the study with students. (Students usually feel more comfortable if this is done in the absence of the guest speaker.) The “Critiquing Criteria” in the textbook (p. 231) can be used to provide the focus for this discussion.
CHAPTER 10 Experimental and Quasiexperimental Designs
Applying Concepts and Principles: Research Design One method of reinforcing previously presented content related to experimental and quasiexperimental designs is to involve students in design construction as a class activity. Present a problem statement and hypothesis for which one of these designs would be appropriate; then walk students through the development of an appropriate design. Students should be allowed to develop any of the design types described in Chapter 10 of the textbook, but should be encouraged to communicate their rationale for the choices they make. After the design has been constructed, discussion should centre on the design’s ability to control extraneous variables, any potential threats to internal and external validity, and alternative approaches to design construction. Here are some suggestions of problem statements (P) and hypotheses (H) to be used: P1: What is the effect of type of educational approach on frequency and perceived self-effi cacy of breast self-examination in adult women? H1: Women who receive live demonstration and supervised practice in breast self- examination conduct the examination more frequently and perceive greater self-effi cacy than those who watch a video presentation. P2: What is the difference in mood state between elderly nursing home residents who receive pet therapy and those who do not? H2: E lderly nursing home residents who receive pet therapy demonstrate more positive mood states than those who do not. 5. Providing Experiential Learning Activities a. Critiquing Experimental and Quasiexperimental Designs This learning activity is designed to help students identify common threats to the validity of fi ndings of experimental and quasiexperimental studies, develop beginning-level skills in critiquing newly learned material, and review previously learned material. Student objectives for the learning activity: Identify potential threats to the validity of study fi ndings. Determine how the research attempted to introduce controls into the study situation to eliminate potential threats to validity. Suggest alternative approaches to deal with potential threats in studies of cause-and-effect relationships. Directions for Implementation. This activity is to be used after students have acquired a basic understanding by reading Chapter 10 and participating in class discussion. Two clinical nursing studies, one that uses an experimental design and one that uses a quasiexperimental design, should be selected. (Suggested studies are provided below.) Students should read these studies before class and be prepared to discuss the following: • Potential threats to internal validity of the studies Ways in which the researcher(s) attempted to control for these potential threats Alternative approaches to either the design or manner in which the researcher(s) dealt with threats to validity. During class, the students should identify the study problem, hypothesis(es), and design. Then you should guide students through critiques of the chosen studies. Suggested studies include the following: Bliss, D. Z., et al. (2001). Supplementation with dietary fi ber improves fecal incontinence. Nursing Research, 50(4), 203–213. (Experimental: prospective, randomized, singleblind clinical trial)
CHAPTER 10 Experimental and Quasiexperimental Designs 53
Goulet, C., Gevry, H., Gauthier, R. J., et al. (2001). A controlled clinical trial of home care management for preterm labour. International Journal of Nursing Studies, 38, 259–269. (Experimental) MacDonald, S. A. (1999). The cardiovascular health education program: Assessing the impact on rural and urban adolescents’ health knowledge. Applied Nursing Research, 12(2), 86–90. (Quasiexperimental) Neu, M., Browne, J. V., & Vojir, C. (2000). The impact of two transfer techniques used during skin-to-skin care on the physiologic and behavioral responses of preterm infants. Nursing Research, 49(4), 215–223. (Quasiexperimental: interrupted, time series, crossover design) Bottorff, J. L., Ratner, P. A., Johnson, J. L., Hislop, T. G., Buxton, J. A., Zeisser, C., et al. (2007). Women’s responses to information on mammographic breast density. Canadian Journal of Nursing Research, 39(1), 38–57. Green, T., Haley, E., Eliasziw, M., & Hoyte, K. (2007). Education in stroke prevention: Effi cacy of an educational counselling intervention to increase knowledge in stroke survivors. Canadian Journal of Neuroscience Nursing, 29(2), 13–20. (Randomized control trial) Ratner, P. A., Johnson, J. L., Richardson, C. G., et al. (2004). Effi cacy of a smoking-cessation intervention for elective-surgical patients. Research in Nursing & Health, 27, 148–161. (Experimental) b. Applying Concepts of Research Design to Evaluation Many health care agencies are increasing staff nurses’ participation in quality assurance activities. This learning activity is designed to provide students with an opportunity to apply knowledge of research design to the delivery of nursing care. Student objectives for the learning activity: Identify a nursing care problem that requires study. Develop a quasiexperimental research design that allows for testing an innovative approach to the problem. Critique the developed design. Directions for implementation. Assist students in identifying a nursing care delivery problem. For example, many health care agencies have found through chart audits that documentation of valid nursing diagnoses is a common problem. Another problem might be the high number of intravenous infi ltrations on a specifi c nursing unit. Once a problem has been identifi ed, ask students to discuss approaches to solving the problem—trying a new technique or system of nursing care delivery. After the class reaches consensus on the approach to be tested, engage students in developing an appropriate research design to evaluate the effi cacy of the new approach. Finally, ask them to critique the research design that they have developed, using the discussion questions in a preceding learning activity. The following are some examples of program evaluation studies that may be helpful in preparing for this activity: Chen, S., Poland, B., & Skinner, H. A. (2007). Youth voices: Evaluation of participatory action research. Canadian Journal of Program Evaluation, 22(1), 125–150. Young, L., Siden, H., & Tredwell, S. (2006). Postsurgical telehealth support for children and family caregivers. Journal of Telemedicine and Telecare, 13, 15–19.
Nonexperimental Designs CHAPTER OVERVIEW Chapter 11 focuses on survey, relationship, and difference studies and on methodological, secondary analysis, and meta-analysis types of research. Relationship studies included are correlational and developmental, that is, cross-sectional, longitudinal and prospective, and retrospective and ex post facto studies. Nonexperimental research designs are explained as those studies that construct a picture or account of events as they naturally occur at one point or over a period of time. In addition to introducing students to types of nonexperimental designs, the chapter outlines the advantages and disadvantages of these designs, the appropriate use of nonexperimental research, and the issues surrounding causality—for example, the fact that nonexperimental research designs preclude the establishment of causal relationships between or among variables. Numerous examples of designs reported in published research are provided. The critique process stresses the appropriateness of the selected design in relation to several factors, namely, the research problem, theoretical framework, hypotheses, methodology, and data analysis and interpretation. This facilitates reinforcement of prior learning and enhances students’ awareness of the interrelated nature of the steps in the research process. LEARNING OUTCOMES After reading this chapter, the student should be able to do the following: Describe the overall purpose of nonexperimental designs. Describe the characteristics of survey and relationship/difference designs. Describe the differences between survey and relationship/difference designs. List the advantages and disadvantages of surveys and each type of relationship/difference design. Identify methodological, secondary analysis, and meta-analysis research. 56 CHAPTER 11 Nonexperimental Designs Identify the purposes of methodological, secondary analysis, and meta-analysis research. Discuss relational inferences versus causal inferences as they relate to nonexperimental designs. Identify the criteria used to critique nonexperimental research designs. Apply the critiquing criteria to the evaluation of nonexperimental research designs as they appear in research reports. Apply levels of evidence to nonexperimental designs.
CHAPTER 11 Nonexperimental Designs 57
TEACHING STRATEGIES Evaluation of the appropriate selection of a nonexperimental research design is in the purview of baccalaureate and master’s students. To perform this evaluation effectively, students must know the rationale, advantages, disadvantages, and criteria for critical appraisal of the design. The teaching strategies that follow will help you to facilitate students’ acquisition of this knowledge. 1. Critical Thinking Challenges The following questions can facilitate student comprehension and critical thinking about this chapter’s content. RECALL AND UNDERSTANDING What is the overall purpose of nonexperimental designs? What are the characteristics of descriptive, exploratory, and comparative designs? What are the characteristics of relationship and difference designs? What are the differences between relational and causal inferences? ANALYSIS AND SYNTHESIS Under what circumstances would you use a nonexperimental rather than an experimental design? What would you consider in deciding whether to use a cross-sectional or a longitudinal design? How would you decide whether to use a retrospective or an ex post facto design? How would you use the information gleaned from a meta-analysis? Involving Guest Speakers It can be extremely helpful to involve guest speakers who have conducted studies using one or more nonexperimental designs. Generally, look to people whose delivery makes their study sound alive and exciting; also try to select studies that are particularly relevant to clinical practice, so that they will have meaning for the students. The guest speaker can provide some background on why the study was conducted and identify the problem, theoretical rationale, and hypotheses; the major focus, however, should be on the design employed and the advantages and disadvantages of its use. Colleagues in education and service can serve as guest speakers. Providing Experiential Learning Activities a. Conducting a Survey The student can learn a great deal about the o perational aspects of a nonexperimental research design by experiencing the process. The purpose of this learning activity is to expand the knowledge that the student brings to the critique of research designs. The descriptive/exploratory survey has been selected as an example of a nonexperimental research d esign because it is relatively easy to implement. Student objectives for the learning activity: Conduct a survey. Summarize survey fi ndings. Evaluate the survey approach to collecting data. Directions for implementation. Students should obtain the following information from 15 people (family members, peers, and others may be included in this sample): Whether they have an annual health examination Their opinions about taking daily vitamins (favourable/unfavourable) How much exercise they get each week (none, some, a great deal) How they estimate their general pain tolerance (low, moderate, high) Their opinions regarding their current health status (poor, fair, good, excellent) Their exact age 58 CHAPTER 11 Nonexperimental Designs Direct students to identify questions that will help them ascertain this information. Once identifi ed, these questions must be asked in exactly the same way for each participant. Findings should be summarized in a two- to three-page paper that addresses the following: How was the information obtained? How long did it take to get the information from each participant? What questions were asked of each participant? What were the fi ndings? Categorize or tabulate these fi ndings according to similarities, differences, and frequency of occurrence among the individuals surveyed. Other ways of organizing the data can be used. Guidelines for discussion. The summary paper, which does not require a grade, can serve as the basis for discussion of the learning activity. The discussion should emphasize both the process and content of the survey. Survey fi ndings can also serve as data for future discussions on descriptive and inferential statistics and sampling. b. Selecting a Nonexperimental Research Design The purpose of this learning activity is to facilitate students’ understanding of how to match the research design to the research question being asked so as to achieve the best fi t between them. Student objectives for the learning activity: Identify the nonexperimental design that is most appropriate for selected research questions. Describe the advantages and disadvantages of the identifi ed design. Differentiate between nonexperimental and other research designs in answering selected research questions. Directions for implementation. Ask students to determine the nonexperimental research design they think most appropriate to answer each of the research questions (1 through 5) listed below. For each research question, they should defend their choice of research design and answer the following questions: Is the design a survey (descriptive/exploratory) or a relationship/difference (interrelationship) study? What are the advantages and disadvantages of using the selected design? How would you rephrase the question to be more appropriate for use with an experimental or quasiexperimental design? RESEARCH QUESTIONS What is the relationship between level of selfesteem and sensation and distress-pain ratings? What types of health services are needed on a college campus? What changes occur in students’ attitudes toward doing homework from Grades 2 through 10? What is the relationship between the incidence of lung cancer in people who live with a smoker and in those who do not? What is the relationship among scores on the Scholastic Aptitude Test (SAT), high school grade average, and success in college? Answer key Correlational Descriptive/exploratory survey Longitudinal Ex post facto Prospective (prediction) This exercise can be implemented in small groups in the classroom or done outside class in the form of a written exercise, either individually or in small groups. The idea is to scrutinize the problem and determine the most appropriate design. Another way to implement this exercise is to use it as a part of an examination. Any of these a pproaches are suitable, and they can be adapted to your own teaching style.
CHAPTER 11 Nonexperimental Designs 59
Guidelines for discussion. Although this learning activity can take place without a follow-up discussion, you may fi nd it helpful to spend time clarifying the relationship between the research question and the research design. This is an excellent opportunity to reinforce the connections among the steps of the research process that have been discussed thus far. c. Critiquing Different (Additional) Research Designs/Types The purpose of this exercise is to provide students with an opportunity to evaluate at least one additional type of research study—that is, one that is different from those presented in Chapter 11 of the textbook. Student objectives for the learning activity: Identify the research design/type of a selected study. Evaluate the research design/type of a selected study. Directions for implementation. This exercise should be conducted after students have read Chapter 11 of the textbook and engaged in relevant classroom discussion on this topic. It is probably unrealistic to expect students to complete more than one critique of a study that exemplifi es an additional type of research. It may be more meaningful for students to complete a critique of a methodological study after they learn the content of Chapter 14 of the textbook, which discusses aspects of tool development and evaluation in greater depth. Suggested studies to use for this exercise are listed below. Have students read the selected study before class and use the criteria in Chapter 11 (p. 250) of the textbook. Students need not write a formal paper but should note their responses to each criterion for discussion in class. As an alternative plan, the class can be divided into three or four groups. Each group can select a different type of study for critique and presentation to the entire class. Suggested studies include the following: Bourque, P., Leger, C., Pushkar, D., & Beland, F. (2007). Self-reported sensory impairment and life satisfaction in older French-speaking adults. Canadian Journal of Nursing Research, 39(4), 154–171. Bryanton, J., Gagnon, A. J., Hatem, M., & Johnston, C. (2008). Predictors of early parenting selfeffi cacy. Nursing Research, 57(4), 252–259. Gocan, S., & Fisher, A. (2005). Ontario regional stroke centers: Survey of neurological nursing assessment practices with acute stroke patients. AXON, 26(4), 8–13. Wells, J., Turner, B., & Coombs, E. (2007). Evaluation of “The many faces of pain”: A chronic cancer pain education program. Canadian Oncology Nursing Journal, 17(2), 91–95.
Sampling CHAPTER OVERVIEW Chapter 12 defines the process whereby representative units of a population are selected for inclusion in a research study. Sampling concepts are introduced in relation to how the population is determined and to the sampling process. Probability and nonprobability sampling strategies are compared and contrasted, the latter being a less rigorous form of sampling. The following nonprobability sampling approaches are discussed in detail in terms of their nature, use, advantages, and disadvantages: convenience, quota, and purposive. Probability sampling approaches described are simple random, stratified random, multistage (cluster), and systematic. Both types of approaches are viewed in terms of their nature, use, advantages, and disadvantages. Also discussed are special sampling strategies, for example, matching and networking (“snowball effect”). Points to consider when critiquing are included for each of the above sampling approaches. Sampling procedures, regardless of which approaches are used, should be systematically organized to reduce bias in sample selection. Sample size is an important factor to consider when evaluating a research report because the size has implications for the generalizability of the findings. The determination of sample size can be a complex process, particularly when probability sampling is used; however, some rules of thumb are presented that should help the novice determine whether sample size is satisfactory to produce meaningful findings. The sampling critique is an important aspect of the methodology of a research study. It should be approached in relation to the parameters or attributes of the study population, representatives of the defined population, appropriateness of the sampling plan to the research design, appropriateness and justification of the sample size, and evidence that the rights of human participants (subjects) have been protected. 62 CHAPTER 12 Sampling LEARNING OUTCOMES After reading this chapter, the student should be able to do the following: Identify the purpose of sampling. Define population, sample, and sampling. Compare a population and a sample. Discuss the eligibility criteria for sample selection. Define nonprobability sampling and probability sampling. Identify the types of strategies for both nonprobability and probability sampling. Identify the types of qualitative sampling. Compare the advantages and disadvantages of specific nonprobability and probability sampling strategies. Discuss the contribution of nonprobability and probability sampling strategies to the strength of evidence provided by study findings. Discuss the factors that influence determination of sample size. Discuss the procedure for drawing a sample. Identify the criteria for critiquing a sampling plan. Use the critiquing criteria to evaluate the Sample section of a research report. CHAPTER 12 Sampling 63
TEACHING STRATEGIES Because sampling concepts and processes can be new and confusing for students, an attempt to simplify this content area is made in the teaching strategies that follow. Through the examination of alternative sampling designs, discussion questions, use of a visual summary, and an experiential learning activity, you can help students put this potentially diffi cult material into a manageable perspective. 1. Critical Thinking Challenges Based on the content of Chapter 12 in the textbook and related lecture material, the following questions can be used to guide a discussion of the sampling process: RECALL AND UNDERSTANDING What is the purpose of sampling? What is the least/most biased sampling d esign? Why? What should you consider before actually drawing a sample? Why is convenience sampling used so frequently in clinical research? What types of sampling are used in qualitative studies? ANALYSIS AND SYNTHESIS What delimitations would you impose in a study of postoperative pain in women who undergo abdominal hysterectomy? Why? What sampling strategy would be most appropriate in studying job satisfaction in nurses? Why? What do you need to consider when obtaining a sample for a grounded theory study? Why is sample size important? How are sampling procedures described in studies you have read? How are generalizability, sample size, and sampling strategy related? 2. Providing Experiential Learning Activities a. Selecting a Sample The purpose of this learning activity is to give student an opportunity to more closely examine the process of sample selection by drawing a simple random sample. Student objectives for the learning activity: Draw a simple random sample, using a table of random numbers (refer to Figure 12-1, p. 267 in the textbook). Evaluate the process of drawing a simple random sample in terms of level of ease or diffi culty and the advantages and disadvantages of this sampling strategy. Directions for implementation. Using the table of random numbers in Chapter 12, students should draw a simple random sample of 20 subjects from one of the following populations: Page of a local telephone directory Roster of the previous year’s graduating class Roster of students currently enrolled in their school or grade level An alternative approach is to divide the class into two or three groups so that several of these populations are used. This will yield a more diversifi ed, interesting discussion. Guidelines for discussion. In addition to discussing the process used to draw a simple random sample, students should be guided in the evaluation of this sampling strategy. For example, ask about how easy or diffi cult and how costly or economical (in terms of time and money) it was to draw the sample, how bias might or might not have been introduced, to whom potential study fi ndings can be generalized, and the major advantages and disadvantages of this approach to sampling. Depending on time and how much reinforcement you think is necessary, you may 64 CHAPTER 12 Sampling wish to discuss alternate sampling strategies for the populations sampled. b. Critical Appraisal Activity The purpose of this learning activity is to give students an opportunity to identify the sampling procedure used in a research report and to critically evaluate the rigour associated with the selected sampling strategy. For this activity you will need the following: Table 12-1 (p. 263) Stewart, M., Reutter, L., Letourneau, N., & Makwarimba, E. (2009). A support intervention to promote health and coping among homeless youths. Canadian Journal of Nursing Research, 41(2), 54–77. Critiquing Criteria (p. 283) Excerpts from the Stewart, Reutter, Letourneau, and Makwarimba (2009) article will be used to provide an example of how to critique this component of a quantitative research article. Responses to the critique questions are not presented in order. Prior to your class, provide a copy of the Methods section of the Stewart et al. (2009) article and ask students to review the Sample and Data Analysis section of the article. The goal is for students to develop a comprehensive understanding of these components of the article and to be able to evaluate the components using the critiquing criteria on page 283. During class, review with students the information provided in Table 12-1 as well as the critiquing criteria. To ensure a comprehensive understanding, review what they understand about the article components prior to moving on to the evaluative phase of the activity, and ask them to review all information related to the sample and data analysis. Create and provide blank copies of a Critical Appraisal Activity sheet and ask students to work together in groups of four to six students to c ritically appraise the Stewart et al. article. As a class, review the correct answers and provide students with ample opportunity to ask clarifying questions. The critical appraisal of the sample and sampling methods requires that the research consumer address 12 important questions: Have the sample characteristics been completely described? Can the parameters of the study population be inferred from the description of the sample? To what extent is the sample representative of the population as defined? Are criteria for eligibility in the sample specifically identified? Have sample delimitations been established? Would it be possible to replicate the study sample? How was the sample selected? Is the method of sample selection appropriate? What kind of bias, if any, is introduced by this method? Is the sample size appropriate? How is it substantiated? Are there indications that the rights of the subjects have been ensured? Do the researchers identify the limitations in the generalizability of the findings from the sample to the population? Are the limitations appropriate? Do the researchers indicate how replication of the study with other samples would provide increased support for the findings? Summary Overview
CHAPTER 12 Sampling 65
The inclusion criteria for the study clearly specifi ed: age 16 to 24 and homeless or in transition from homelessness. No sample delimiters were identifi ed. Given the population being studied, the sampling method used (convenience sampling) was appropriate for the pilot. If this had been a full randomized control trial, network sampling would have been more appropriate. Given the chosen sampling method, it would be extremely diffi cult to replicate this sample (Questions 4, 5, 6, 7). Seventy individuals agreed to participate and completed the pretest; however, only 56 people participated in the intervention. A sample size of 56 was inappropriate given that a power analysis suggested that the minimum sample size required was 70 (Question 9). Under the Methods section (p. 60), the researchers document how the rights of participants were ensured: voluntary participation, confi dentiality, freedom to withdraw (Question 10). The sample characteristics were described in great detail under the section heading Profi le of Participants. The profi le offers information on the age, race/ethnicity, parental status, level of education attained, employment status, living a rrangements, and composition of their social network (Question 1). It remains unclear if the sample is representative of the general homeless youth population and, given the sample size, the parameters of the study population cannot be inferred. We do know for certain that aboriginals were oversampled and that this may have infl uenced participation, retention, and the study outcomes (Questions 2, 3, 8). The researcher very briefl y mentions attrition as a major challenge but does not address this in terms of generalizability (Question 11,12). Given the sample size, sampling method, attrition, lack of representation, and measurement issues, these fi ndings cannot be generalized to the larger homeless youth population. The randomized control trial would have to be altered signifi cantly to be valid. However, it is recognized that this is a pilot study. Instructor Manual for Nursing Research in Canada: Methods, Critical Appraisal, and Utilization Geri LoBiondo-Wood, Judith Haber, Cherylyn Cameron, Mina Singh 9781926648545, 9781771720984, 9780323447652, 9780323057431
