Preview Extract
This document contains Chapters 15 to 16 CHAPTER 15 USING MULTIVARIATE DESIGN AND ANALYSIS QUESTIONS TO PONDER 1. What statistics are used to evaluate correlational and experimental multivariate relationships? 2. What are the key assumptions and requirements of multivariate statistics? 3. How do various violations of the assumptions underlying multivariate statistics affect your data analysis? 4. When is factor analysis used, and what do factor loadings tell you? 5. Why are factors rotated in factor analysis? 6. What is the difference between principal components and principal factors analysis? 7. When do you use exploratory or confirmatory factor analysis? 8. What do partial and part correlations tell you? 9. When would you use partial correlation? 10. When would you use part correlation? 11. For what research applications would you use the various types of multiple regression analysis? 12. How are multiple R, R 2 , and adjusted R 2 used to interpret the results from a multiple regression analysis? 13. What is the difference between the raw and standardized regression weights, and why are the standardized weights used when interpreting the results from a regression analysis? 14. What is the squared semipartial correlation, and when is it used? 15. What is discriminant analysis, and when is it used? 16. What do discriminant functions tell you? 17. What are the two main applications of discriminant analysis? 18. What is canonical correlation, and when is it used? 19. When would you use a multivariate analysis of variance (MANOVA) to analyze your data? 20. How are the results of a MANOVA interpreted? 21. Why would you use a MANOVA to analyze data from a within-subjects design? 22. What is multiway frequency analysis, and when is it used? 23. What is loglinear analysis, and when is it used? 24. What is G2 , and how is it used in loglinear analysis? 25. What is path analysis, and how is it used in the research process? 26. Why is it important to develop a causal model when using path analysis? 27. What are the different types of variables used in a path analysis? 28. How are path coefficients used to interpret a path analysis? 29. What is structural equation modeling, and how does it diff er from path analysis? 30. What are the three confirmatory applications of structural equation modeling? 31. What is a latent variable, and how is it used in structural equation modeling? 32. Why should you exercise caution when using multivariate data analysis? CHAPTER OUTLINE Correlational and Experimental Multivariate Designs Correlational Multivariate Design Experimental Multivariate Design Assumptions and Requirements of Multivariate Statistics Linearity Outliers Normality and Homoscedasticity Multicollinearity Error of Measurement Sample Size Correlational Multivariate Statistical Tests Factor Analysis Partial and Part Correlations Multiple Regression Discriminant Analysis Canonical Correlation Experimental Multivariate Statistical Tests Multivariate Analysis of Variance Multiway Frequency Analysis Multivariate Statistical Techniques and Causal Modeling Path Analysis Structural Equation Modeling Multivariate Analysis: A Cautionary Note Summary Key Terms
univariate strategy multivariate strategy multivariate design multiple regression discriminant analysis canonical correlation factor analysis multivariate analysis of variance (MANOVA) partial correlation part correlation multiple R R -square beta weight multiway frequency analysis loglinear analysis path analysis structural equation modeling (SEM) latent variable
CHAPTER GOALS Chapter 15 provides an overview of multivariate statistical analysis. Our goal in writing this chapter was to introduce students to the major multivariate statistical procedures, including when they would be used, what they would be used for, and what the underlying logic is. Given the increasing use of multivariate statistical techniques, we feel that students should at least have some idea of what they are, and why they are sometimes used in place of univariate statistics. It was not our intention to provide a thorough grounding in multivariate analysis. In keeping with this goal, our coverage of multivariate techniques is broad rather than deep. The chapter begins by listing the most common correlational and experimental multivariate designs and explains the advantages of multivariate designs over univariate ones. This is followed by a discussion of the assumptions and requirements of multivariate statistics (linearity, outliers, and how to deal with them; normality, homoscedasticity, multicollinearity, the issues of measurement error, and sample size). The remainder of the chapter introduces specific multivariate techniques. For each technique, we indicate what the technique is used for, describe the major issues a researcher must deal with when using the technique, and provide an example of its use. Key points to bring up in classroom discussions include the following: 1. The requirements and assumptions of multivariate statistics. You should define each requirement/assumption, and discuss how important it is that these be met. 2. The idea that in factor analysis the goal is to identify a small number of underlying variables (factors) that determine the scores on the measures given. The scores obtained from each measure reflect the combined effects of two or more underlying variables, and factor analysis uses the pattern of intercorrelations between measures to “extract” the underlying factors. 3. Part and partial correlations as attempts to statistically control the effects of extraneous “third” variables to determine how two variables would be related if the third variable were held constant. This offers a possible solution to the “third variable problem” that, along with the directionality problem, makes inferring cause from correlation dangerous. 4. The fact that, in multiple regression, multiple R can be interpreted simply as the ordinary Pearson correlation between the actual values of the criterion variable and the values predicted by the linear regression equation. You might want to explain how the order in which variables are entered into the analysis affects the weights assigned to the variables when the predictor variables are intercorrelated. Attempts to deal with this problem lead to the different types of regression analysis (simple, hierarchical, and stepwise). 5. Reinforcement of the main distinction between discriminant analysis and multiple regression, which is that discriminant analysis is used when the criterion variable is categorical (nominal scale), whereas multi-pie regression is used when the criterion variable falls on an interval or ratio scale. 6. The fact that canonical correlation is similar to multiple regression except that the criterion “variable” is a linear combination of variables similar to the linear combination of predictor variables. 7. The use of MANOVA when you have multiple dependent variables in a design that otherwise would call for ANOVA. The MANOVA helps avoid the probability pyramiding that would occur if each dependent variable were tested separately. Moreover, it can identify combinations of dependent variables that may be sensitive to the manipulation of the independent variables when each dependent variable by itself is not sensitive enough to give significant results. 8. The role of MANOVA in the analysis of data from within-subjects designs when you suspect that the homogeneity of covariance assumption required for a within-subjects ANOVA may be violated. Rather than assuming equal covariances, MANOVA takes the observed covariances into account when determining statistical significance. 9. That loglinear analysis provides an alternative to traditional analyses in both correlational and experimental situations in which the dependent variables were categorical. It can also be used as a nonparametric alternative when the assumptions of parametric tests are suspected to have been violated. 10. That path analysis is used to construct causal models. Describe how path analysis works. Perhaps have students evaluate an actual example of path analysis from the literature. IDEAS FOR CLASS ACTIVITIES One way to get students acquainted with multivariate data analysis is to describe a study that used a multivariate approach, and then guide the students through the results of the statistical analysis. This can be particularly effective if your example comes from your own research, and you can provide students with copies of the analysis as printed out by one of the major statistical packages such as SPSS. Avoid lengthy explanations of eigenvalues or Wilks’s Lambda; instead, focus on the conclusions you were able to draw from the analysis. Lens Model Study This correlational study uses multiple correlation to determine the weighting a subject gives to various sources of information that are used to make a decision. According to the “lens model,” a participant’s judgment about something like the desirability of a particular political candidate depends on many factors (termed “cues”), such as the candidate’s party affiliation, position on the liberal–conservative scale, ability to communicate, religious views, and physical attractiveness. These characteristics are somehow combined to form the judgment. If we assume that the participant is forming a judgment by, in effect, weighting each cue according to its importance and then adding the weighted cues, we can model the judgment using multiple regression. Multiple regression will find the set of weights that maximize the correlation between the actual judgments and the judgments predicted by the regression equation. The size of the multiple R, then, indicates the degree to which the participant’s judgments can be predicted by a linear combination of the cues. You can determine which cues affect the participant’s judgments most strongly by analyzing the regression weights assigned to the cues. Students usually enjoy this exercise even though they do not fully understand how multiple regression works. After all, it tells them something interesting about themselves! The study involves having the class decide on something to make a judgment about and on the cues that will be given in order to form a decision. For example, they might want to determine how people judge the desirability of an automobile. The class might then debate which sources of information need to be included: styling, performance, fuel efficiency, color, interior roominess, and so forth. Eventually they should narrow the field of cues down to about five cues. The next step is to create a measurement scale for each cue. These should be numerical rating scales with verbal anchors. For example, styling could be rated from 1 (ugly) to 7 (knockout). A similar scale needs to be created for the judgments, perhaps from 1 (you couldn’t GIVE it to me) to 7 (I’d sell my soul for it). All the scales should be placed on a 3-inch x 5-inch card that the subject can refer to while making the judgments. Now that the cue and judgment scales have been established, it is time to create a profile sheet. This sheet might look something like the following: PROFILE NO. ____ Styling _____ Performance _____ Fuel Efficiency _____ Safety Factor _____ Roominess _____ Rating _____ Reproduce enough of the rating sheets to create 100 profiles for each member of the class. Number the profiles from 1 to 100, and fill in the cue values by random assignment. Leave the space labeled “Rating” blank. Each student should be given a packet of 100 profiles. The students should be instructed to carefully examine each profile and try to imagine what the car (or whatever) being rated would be like based on the description provided by the ratings. The student should then fill in a rating for the profile based on this impression. Analysis Use a computerized statistical program to perform a multiple regression analysis of each student’s ratings. This analysis should provide the multiple R, simple correlations between each pair of variables, standardized regression weights (beta weights), and, if possible, the squared semipartial correlations. Use the cue values as predictor variables and the ratings as the criterion variable. Obtain a printout of the analysis for each student, and give it to the student. The example given in the text to illustrate multiple correlation is in fact a study like the one described here. In that example, we describe how to analyze and interpret the data including how to calculate squared semipartial correlations if need be. However, if the profiles have been constructed by choosing scale values at random, correlations between predictor variables will usually be low and in that case the beta weights themselves usually lead to conclusions similar to those based on the squared semipartial correlations. The variables given the largest weights will be those the student relied on most heavily to make a decision (assuming the linear model is valid). The size of R-square indicates the degree to which the linear model is able to predict the subject’s judgments from the cue values. CHAPTER 16 REPORTING YOUR RESEARCH RESULTS QUESTIONS TO PONDER 1. How do you set up a paper using APA writing style? 2. What is the heading structure used in an APA-style manuscript? 3. What information is included on the title page, and in what order would you find that information (from the top of the page to the bottom of the page)? 4. What is an abstract, and why is it so important? 5. What information goes into an abstract, and how long should an abstract be? 6. What information is included in the introduction to an APA-style paper? How is the introduction organized? 7. What information would you expect to find in the method section? 8. Describe the various subsections of the method section. 9. What would you expect to find in the results section of a manuscript? 10. How is the results section formatted, and how are statistics reported? 11. How is the discussion organized, and what would you expect to find in the discussion section? 12. Where do you begin the reference section in an APA-style manuscript? 13. What information do you include in an APA-style reference, and how is a reference entry formatted? 14. How are footnotes used in an APA-style manuscript, and where are they placed? 15. When are tables used in an APA-style manuscript? 16. How are the tables used in an APA-style manuscript formatted? 17. When do you use figures in an APA-style manuscript? 18. How is a page containing a figure set up, and what is included on a figure page? 19. What are the general rules for in-text citations? 20. How do you cite quoted material in your paper? 21. What are the general rules for using numbers in the text of a manuscript? What are the main exceptions to the general rules for using numbers? 22. What is biased language, and why should you avoid using it? 23. What are the three APA guidelines for avoiding biased language? 24. Why are precision and clarity of expression, organization, and style so important to consider when preparing a manuscript? 25. What factors contribute to or detract from precision and clarity of expression? 26. What factors contribute to or detract from good organization? 27. What can you do to ensure that your paper has proper style? 28. What are plagiarism and lazy writing? 29. How can you avoid plagiarism and lazy writing? 30. What is typically the sequence of events involved in submitting a paper for publication? 31. What are an oral presentation and a poster session, and how do they differ? 32. What are the ethical obligations involved in reporting or publishing your results? CHAPTER OUTLINE APA Writing Style Writing an APA-Style Research Report Getting Ready to Write Parts and Order of Manuscript Sections The Title Page The Abstract The Introduction The Method Section The Results Section The Discussion Section The Reference Section Footnotes Tables Figures Elements of APA Style Citing References in Your Report Citing Quoted Material Using Numbers in the Text Avoiding Biased Language Expression, Organization, and Style Precision and Clarity of Expression Economy of Expression Organization Style Making It Work Avoiding Plagiarism and Lazy Writing Telling the World About Your Results Publishing Your Results Paper Presentations The Ethics of Reporting or Publishing Your Results Summary Key Terms
running head title page author note abstract introduction method section participants subsection subjects subsection apparatus subsection materials subsection procedure subsection results section discussion section reference section biased language plagiarism lazy writing
CHAPTER GOALS Chapter 16 introduces students to writing research reports in APA style. The chapter begins by introducing students to the basics of APA writing style (we could not hope to cover all aspects of APA writing style in a single chapter). The student should learn how to set up an APA-style paper, including all of the necessary sections from the title page to the references. A section-by-section analysis tells what goes into each section and subsection, describes how to set up a table for data presentation, and shows how to properly prepare graphs. The second major section of the chapter emphasizes the importance of clarity of expression, organization, and style. Students often do not pay enough attention to good writing practices and produce papers written in stilted “scientificese” (e.g., incomplete sentences describing a procedure). Reinforce the importance of constructing grammatically correct sentences and organizing those sentences into coherent paragraphs. Paragraphs should be organized into larger coherent sections. Also, emphasize to students the importance of handing in papers that have been adequately proofread and corrected. A section on plagiarism and lazy writing is presented to help students avoid those serious flaws in writing. Finally, a brief section on the process involved in disseminating research results in journals, paper presentations, and poster sessions is included. In class you should highlight the following points: 1. How to set up an APA-style paper (proper margins, major sections, and so on). 2. How to format the title page and abstract within APA guidelines. Emphasize that the abstract is a concise summary of the reported research. Spend some time on the abstract—in our experience students have a great deal of trouble with it. 3. How to write an effective introduction. Highlight the general-to-specific organization of an introduction. Also reinforce the importance of a thorough, yet concise, literature review and a clear statement of the research hypothesis. 4. What information goes into each subsection of the method section. Emphasize that the information should be complete enough so that someone could replicate the study. Remind students to write in complete sentences in the method’s section. 5. How statistics are reported in the result’s section and how data can be presented in tables or graphs. 6. How a discussion section is organized from specific to general, beginning with a restatement of the research hypothesis and ending with broad implications of the findings. 7. How to format references both in the body of the report and in the reference section. Emphasize that any citation appearing in the text must be included in the reference list. Remind students about properly citing secondary sources. 8. How to order references in the reference list. 9. The importance of clarity of expression, organization, and style. It does not hurt to keep reminding students that they should pay attention to grammatical correctness, proper word choice, and economy of expression. Additionally, point out the importance of organizing sentences and paragraphs coherently. 10. The issue of biased language and how to avoid it. Remind students that they should proofread their writing and eliminate instances of biased language. 11. Define plagiarism and lazy writing, and give examples of each. Convey that both are unacceptable. 12. The ways that research is disseminated via publication and presentations at professional meetings. IDEAS FOR CLASS ACTIVITIES Student Critiques Students often have difficulty communicating their ideas clearly on paper, yet are unable to identify the flaws in their writing. This exercise allows students to critique each other’s work and can be carried out on lab reports or full APA-style papers. Have students prepare and hand in two copies of a lab report or a full APA-style paper based on one of the research exercises carried out during the course. Remove the title page from each copy and assign each paper a code number (perhaps the last four digits of the student’s student ID number) so that you can return the papers to their rightful owners. This technique amounts to a “masked review.” Students may feel less inhibited about evaluating another person’s work if the whole process is anonymous. Distribute the papers to the class, making sure that no student receives his or her own paper. Have students evaluate and correct the papers by answering the following questions: 1. Are all of the sections and subsections of the paper in the proper order? 2. Within each section, is all of the necessary information present (e.g., are the subjects, materials, and procedures described fully)? 3. Within each section, is the information presented in an organized, clear way? Are the paragraphs connected to one another logically? Are there any confusing, poorly written, or ambiguous sentences? 4. Are there any stylistic errors (e.g., uncorrected typographical errors, misspelled words, improperly used words, glaring grammatical errors)? 5. Is there any evidence of biased language? 6. Are all the references given in the reference section cited in the paper? Conversely, are all references cited in the paper given in the reference section? Poster Presentation As a substitute for (or in addition to) preparing a written lab report or paper, have each student (or group of students if they work in groups) prepare a poster to present the results of one of the research projects done during the course. The posters should include an abstract, an introduction, and method, results, discussion, and reference sections. Have the students give a brief oral presentation of their project to the class. Uncovering Biased Language Assign students to find one or two articles in the library that contain examples of biased language. In class, have students rewrite the biased sections to eliminate the bias. If you can obtain a computer with a projection screen, you can actually rewrite the biased sections in front of the class to incorporate their suggestions. One way to do this exercise is to have students find articles from the 1960s, l970s, 1980s, 1990s, and 2000s from the same journal. Have them content-analyze the articles for biased language. Do they note any changes in how racial, religious, ethnic, or gender groups are referred to? Building an APA-Style Paper To illustrate how an APA-style paper should be written, you can use a class meeting to build a paper. Use one of the research projects conducted during the semester, and have students contribute ideas about how each major section should be written. This exercise will work best if you use a computer with a projection screen. Type in the suggestions, and then edit the text for maximum clarity and completeness. Using Internet Resources A variety of sources of information on APA writing style are available on the Internet. For this exercise, have students find some of the resources available on the Internet to help them write APA-style papers. [But warn them that some of these cites present information from older editions of the APA Publication Manual and may have changed for the current (6th) edition.] Perhaps the best resource for APA style (6th ed.) is the APA’s website devoted to that topic. You can access the home page at http://www.apastyle.org/manual/index.aspx . From there students can follow links to chapter descriptions, tutorials, and other resources. Another excellent resource for learning APA style is Purdue University’s OWL (Online Writing Lab) page on that topic, which has been updated for 6th edition. You can access it at http://owl.english.purdue.edu/owl/resource/560/01/ . Instructor Manual for Research Design and Methods: A Process Approach Kenneth Bordens, Bruce Barrington Abbott 9780078035456
