CH-5-9 Secondary Sources of Marketing Data – Marketing Research : Chapter Notes

This Document Contains Chapters 5 to 9 PART II SECTION A TEACHING NOTES FOR CHAPTERS CHAPTER FIVE SECONDARY SOURCES OF MARKETING DATA Learning Objectives • List the various secondary data sources. • Discuss the uses of secondary data. • List the benefits and limitations of secondary data. • Describe the internal sources and forms of secondary data. • Explain the value of demographic, economic, and social statistics contained in census data. • Explain the importance of the North American Industry Classification System. • Discuss how to appraise secondary sources. • Describe applications of secondary data. • Discuss the sources of secondary data used in international marketing research. • Explain the problems associated with secondary data in international marketing research. • Discuss applications of secondary data in domestic and international marketing research. Teaching Suggestions In one important sense this chapter and the next one represent a transition, from the introductory planning and design issues, to the specific activities of collecting and interpreting data. The objectives for this chapter should be: a) Introduce students to the wealth of available data so they can see the necessity of beginning any research study with the pursuit of readily available secondary data. b) To inform students about internal sources of data, that can be leveraged for more firm specific information and insights c) To understand how secondary data can be used to provide answers to marketing questions. d) To provide students with information on the NAICS system that has replaced the old SIC system. e) To recognize the limitations of secondary data. Of course, these skills will be enhanced in subsequent chapters which discuss the collection of primary data. Knowledge of the difficulties of collecting data in general will give a further appreciation of the deficiencies of secondary data. In general, the rule for using secondary data should be: don’t ignore it, but don’t rely on it completely. In addition to the questions for discussion, some useful devices for achieving the objectives of this chapter are to explain the new North American Industry Classification System (NAICS) to the students, highlight its benefits and to briefly refer to the problems of the old SIC system. A brief FAQ on the NAICS is provided in the next page for instructor’s review. Using the team projects as a vehicle to familiarize students with the new NAIC system, instructors could provide the Web addresses under the “Resources to Students” section of the FAQ as further sources of information about the differences between the SIC and NAIC systems. Of particular importance is Figure 5-1, which is an excellent resource for understanding the process of secondary data collection. It is also important to understand the importance, benefits, sources, and applications of secondary data research. In light of the current advances in technology applications in the area of marketing research, the need for discussing computer retrievable databases becomes important. The instructor may also wish to discuss census data (Figure 5-3 explains the geographic subdivisions of a MSA) NAICS AND SIC The North American Industry Classification System (NAICS) has officially replaced the U.S. Standard Industrial Classification (SIC) system. The NAICS provides a consistent system for economic analysis across the three North American Free Trade Agreement partners – Canada, Mexico and the United States. The system was developed by the U.S., Canada, and Mexico to provide comparable statistics across the three countries. For the first time, government and business analysts will be able to compare directly industrial production statistics collected and published in the three North American Free Trade Agreement countries. NAICS also provides for increased comparability with the International Standard Industrial Classification System (ISIC, Revision 3), developed and maintained by the United Nations. NAICS industries are identified by a 6-digit code, in contrast to the 4-digit SIC code. The longer code accommodates the larger number of sectors and allows more flexibility in designating subsectors. It also provides for additional detail not necessarily appropriate for all three NAICS countries. The international NAICS agreement fixes only the first five digits of the code. The sixth digit, where used, identifies subdivisions of NAICS industries that accommodate user needs in individual countries. Thus, 6-digit U.S. codes may differ from counterparts in Canada or Mexico, but at the 5-digit level they are standardized. The new NAICS hierarchical structure is: XX Industry Sector (20 broad sectors up from 10 SIC) XXX Industry Sub-sector XXXX Industry Group XXXXX Industry XXXXXX U.S., Canadian, or Mexican National specific NAICS provides a consistent framework for the collection, analysis and dissemination of industrial statistics used by: Government policy analysts Academics and researchers Business community and Public. NAICS will be reviewed every 5 years so that classifications and information keep up with the changing economy. Key points to note about NAICS Summary of NAICS and how it is used The North American Industry Classification System (NAICS, pronounced Nakes) was developed as the standard for use by Federal statistical agencies in classifying business establishments for the collection, analysis, and publication of statistical data related to the business economy of the U.S. NAICS was developed under the auspices of the Office of Management and Budget (OMB), and adopted in 1997 to replace the old Standard Industrial Classification (SIC) system. It was also developed in cooperation with the statistical agencies of Canada and Mexico to establish a 3-country standard that allows for a high level of comparability in business statistics among the three countries. NAICS is the first economic classification system to be constructed based on a single economic concept. Assignment of NAICS codes to businesses There is no central government agency with the role of assigning, monitoring, or approving NAICS codes for establishments. NAICS Codes are self-assigned based on the establishment’s primary activity (the activity that generates the most revenue for the establishment). Individual establishments are assigned NAICS codes by various agencies for various purposes using a variety of methods. The U.S. Census Bureau has no formal role as an arbitrator of NAICS classification. The U.S. Census Bureau assigns one NAICS code to each establishment based on its primary activity (the activity that generates the most revenue for the establishment) to collect, tabulate, analyze, and disseminate statistical data describing the economy of the United States. Generally, the U.S. Census Bureau’s NAICS classification codes are derived from information that the business establishment provided on administrative, survey, or census reports. (e.g. when a company applies for an Employer Identification Number (EIN), information about the type of activity in which that business is engaged is requested in order to assign a NAICS code). Various other government agencies, trade associations, and regulation boards adopted the NAICS classification system to assign codes to their own lists of establishments for their own programmatic needs. Market-Based vs. Production-Based statistical classifications and NAICS A new North American Product Classification System (NAPCS) is under development, starting in nine service sectors. Whereas NAICS focuses on the input and production processes of industries, NAPCS will classify all the output of the industries of NAICS. The long-term objective of NAPCS is to develop a market-oriented, or demand-based, classification system for products that (a) is not industry-of-origin based but can be linked to the NAICS industry structure, (b) is consistent across the three NAICS countries, and (c) promotes improvements in the identification and classification of service products across international classification systems, such as the Central Product Classification System of the United Nations. NAICS codes with 5-digits NAICS is a two-through-six-digit hierarchical classification code system, offering five levels of detail. Each digit in the code is part of a series of progressively narrower categories, and the more digits in the code signify greater classification detail. The first two digits designate the economic sector, the third digit designates the subsector, the fourth digit designates the industry group, the fifth digit designates the NAICS industry, and the sixth digit designates the national industry. A complete and valid NAICS code contains six digits. In developing NAICS, the United States, Canada, and Mexico agreed that the 5-digit codes would represent the level at which the system is comparable among the three countries. The sixth digit allows for each of the countries to have additional detail (i.e., subdivisions of a 5-digit category). In cases where the U.S. did not choose to create additional detail, the 5- and 6-digit categories within U.S. NAICS are the same, and the 6-digit US NAICS code ends in zero. In some hierarchical presentations, the 6-digit code is omitted where it is the same as the 5-digit category. NAICS Codes with more than 6 digits Industries in the United States are identified by a six-digit code. Each manufacturing and mining product or service is identified by a ten-digit product code. The product coding structure represents an extension, by the U.S. Bureau of the Census, of the six-digit industry classification of the manufacturing and mining sectors. The product classification system operates so that the industrial coverage is progressively narrower with the successive addition of digits. More information about NAICS can be found at www.naics.com The instructor may also wish to discuss to sources of secondary data in international marketing research. The problems encountered in secondary data collection in the international context, data comparability, and applications are areas that could be discussed in this section. Questions and Problems 1. A variety of useful secondary data is available to someone deciding where to locate a store to sell mobile devices, accessories and wireless connections. From studies, it should be possible to determine the profile of the target market. The next step is to identify and locate the potential competitors in the broad geographic area. Any new entrants will also need to be identified. Supplier salespeople are a good source. The location should be as far away from competitors as possible and convenient to large concentrations of buyers. The latter assessment can be aided with Census data on manufacturing and service establishments in census enumeration areas, coupled with a directory of local businesses. A service like PRISM could be used to identify zip code areas with reasonable concentrations of customers in the target category. While the logic is the same when locating a pizza outlet, the analysis should take into account the difference in shopping behavior - and category specific differences. The profile of the target markets will be different, for example, areas around businesses and universities and colleges are attractive markets. It should also be desirable to have data on traffic, density and movement which could be provided by the local planning authority or urban transit agency. 2. The first question is; what kinds of secondary data should be available for the market area of each proposed location? Relevant data for the area would include: a) buying power as a function of disposable income and population, b) mix of apartment versus single home dwellers in the population. c) economic health of the area, including plans for expansion or contraction of the area, presence of universities, offices etc., d) Competitive presence, performance (i.e. ability to satisfy present needs of area) and future plans; e) traffic counts and likely changes in road networks that would affect traffic passing near the location. 3. Industry associations are one method of obtaining direct data. There are consulting firms such as Frost and Sullivan who conduct industry studies for clients. These types of firms can be contacted for industry information. In order to answer this question, students will either have to contact firms in each of the six industries and ask for the names of the industry associations or go to the library. The associations for each industry can then be ranked by the students in terms of their usefulness for providing secondary industry data. Students might also be asked to contact several industry associations to see the type of secondary data that is available. 4. Data on cigarette consumption can be obtained-through a combination of government services, industry associations and by contacting firms in the industry. The difference between per capita consumption in a particular state or province versus the country as a whole can be explained by demographic variables. 5a. Since Educational Edge is a small company with limited resources, assessing market potential through a marketing research study is out of question. Also the product is a not an innovative product and hence there should be no dearth of secondary data. Secondary data are data collected by persons or agencies for purposes other than the solution of the problem at hand. The examination of the secondary data may provide enough information to actually resolve the problem investigated. Therefore, it is better for Educational Edge to go in for secondary data to assess the market potential for the erasable transparencies. In case, any new idea is generated, it can be tested later through primary research. The bulk users of erasable transparencies would be in Schools, Universities and Corporations. While schools and universities will use the transparencies for teaching and presenting purposes, the corporations will tend to use transparencies predominantly for presentation purposes. Educational edge will be interested in obtaining a list of educational institutions and corporations so that it can devise a strategic plan and decide on the market segments. Educational edge can obtain the relevant information about the educational institutions from government published sources. The number of teachers in any geographic segment can be obtained from the census data. Using rule of the thumb and experts’ advice, the number of transparencies per teacher for a time period is calculated, multiplied with the total number of teachers and hence the potential in the educational sector for a time period is calculated. These figures can be cross-checked using alternate methods. Educational edge can identify a competitor and arrive at the competitor’s sales from their annual report. Using the competitor’s market share, the market potential can be calculated. Also, trade association journals can give an idea of the major players, their share, market growth rate and market potential. Similarly, NAICS related data could help Educational Edge to assess the industrial market potential. b. Market segmentation is required by businesses seeking to improve their marketing efforts. Effective segmentation demands that firms group their customers into relatively homogeneous groups. Educational edge can use the North American Industrial Classification System (NAICS) and the Dun’s market identifiers (DMI) to segment its industrial market. c. By using secondary data, Educational edge can initiate a lot of savings in cost, effort and time. In this case, it would neither be physically nor financially possible for Educational edge to conduct a primary research to obtain data. The limitations are that the information received from published sources could have become dated and hence problems of fit could arise. Also, the accuracy of data and its error bounds are not known. There is a possibility that some assumptions may have to be made when the data is used for purposes of calculation. 6a. Howard Enterprises should opt for secondary data to select countries that merit in-depth investigation. This is because Howard is a small company with limited resources and a marketing research study would cost money. Also, primary data collection, in the international context, is time consuming. Alternatively, secondary data is relatively inexpensive and can be used to arrive at interesting propositions that can be tested later through primary research. Secondary data sources can help Howard to assess market opportunities in countries that it has little knowledge. For the initial part of identifying the countries, a wide variety of secondary data sources can be used. These range from sources that provide general economic, social and demographic data for almost all the countries in the world. For preliminary screening of countries, Howard could use macroeconomics data, based on various sources like United Nations and World Bank. Similarly, Howard could also look at industry specific data for market opportunity and analysis. Also, sources specific to individual countries or product markets should be looked into. For instance, if Howard feels that there is ample potential for lamps in the United Kingdom, it could consult the International marketing handbook, published by the US department of Commerce which provides profiles and special information about doing business in the UK Also, the UK consulate trade office will be in a position to explain more about the investment opportunities in there. Howard should also check for private published sources to obtain industry relevant information. b. Howard should keep in mind that data acquired from secondary sources, in the international context has two major limitations. Comparability and accuracy of the data: Various sources report various values for the same indicators due to different definitions followed for the indicators in those countries. For instance, a huge corporation in Nepal will be taken as a medium sized company in Australia. Hence the comparison of statistics creates a big problem in the international context. Also, the accuracy of the data is also highly variant between countries. 7. From the EIS Industrial Plants or EIS Manufacturing Establishments obtain information about any one industry. This can be done by selecting the NAICS code for that industry. EIS provides information on the value of the shipments, share of the market estimates and other information about organizations. Another way to obtain the same information is to collect the sales figures for the brands in this product category from the retail stores. Because only a select number of retail stores will be contacted and the sales for the product will be extrapolated from this, this can give a different sales estimate. PART II SECTION A TEACHING NOTES FOR CHAPTERS CHAPTER SIX STANDARDIZED SOURCES OF MARKETING DATA Learning Objectives • Describe the process of conducting retail store audits. • Discuss the advantages and limitations of consumer purchase panels. • Discuss the sources and applications of scanner data. • Describe the value of commercial information collected from media-related standardized sources. • Describe a framework for the various applications of standardized data sources. Teaching Suggestions This chapter builds on Chapter 5 and discusses another alternative to primary sources of marketing data available to the marketing manager. Two major sources of marketing data have developed fairly recently in recent years—computer retrievable data and scanner-based research services. These have been the result of advances in computer and information transmission technologies. Discussion of scanner-based research and single source systems can be used to illustrate the quantity of marketing data available to the manager. This can be balanced against his need for information rather than volumes of raw data. Using this perspective, the necessity for effective marketing decision support systems can be underscored. These can be seen to provide the framework for effective use of both primary and secondary sources of marketing data. Questions and Problems 1. A product audit would be much more suitable for cameras and engine oil additives which are sold through a variety of specialty outlets and have long repurchase cycles. The major concern is with the extent of distribution coverage. By contrast, the Nielsen store audit is preferred for peanut butter because it is a frequently purchased product sold through grocery stores. The primary concern is with monitoring sales fluctuations and competitive activity within this setting. Finally, chewing gum sales and distribution coverage could be handled with a combination of methods. The difficulty with this product is the enormous array of convenience outlets that sell the product. These outlets are not fully covered by Nielsen audits. 2. This problem illustrates the way managers can use scanner data to analyze the effectiveness of their advertising vehicles. It is drawn from an actual experiment conducted for a Midwest retailer by TRIM inc., the Los Angeles-based market research firm that evaluates scanner data for retailers. (Chain Store Age/Supermarkets, June 1983). This ad is also reproduced below. Identical advertisements were run in four competing newspapers. Scanner data was monitored for stores located in areas serviced primarily by each newspaper. The products targeted in the advertisements were both food and non-food items and, in addition, the promotional appeals were varied. Scanner printouts from the specific market serviced by each newspaper indicated their selling effectiveness, by general product category and type of promotion. Effectiveness was measured by the percentage change in the average number of units sold per 1,000 shoppers in the store during the week the advertisement appeared. Advertising policies could then be based on an evaluation of effectiveness relative to costs for each type of advertisement and for different product classes, for each newspaper. The manager of one store, as in this problem, could conduct a similar experiment by placing identical advertisements sequentially in the two newspapers, offering promotions redeemable with the coupon in the advertisement. Scanner data for the advertised product would be collected for sales before, during and after the appearance of the advertisements. Other possible experiments noted in the text are different combinations of advertising copy, discount coupons, in-store discounts, and promotions. 3. The Nielsen Store Audit Report could be used to measure the sales and distribution figures for the new cereal brand vs. the company's existing cereal brands. The type of information yielded by this report is listed in Table 6-1. Students should realize that distribution levels of the new brand would be critical to evaluating its impact on current cereal brands. Presumably the new brand would be launched with media and retailer support and this report would help the manager track retail displays for the new product. The SAMI report could be used in combination with the Nielsen report to track the cannibalization of sales of the current cereal brands by the new brand at the wholesale level. Consumer Purchase Panels would be the best way to monitor trial and repeat purchase of the new cereal product, since these services would enable the manager to disentangle trial from repeat purchases. This information is not available by simply looking at product movement through the distribution channel. Scanner-based Audit Services could also be used. 4. Consumer Panels may underestimate brand sales and market share in comparison with a scanner audit for certain types of products purchased by multiple household members. Since usually only one person in the household records purchases, items bought by the members can be missed. Scanners on the other hand record all purchases made in a retail store regardless of which household member does the actual buying. A scanner audit will be more accurate and consumer panels may be supplemented by scanner audits to estimate the proportion of variance between the two methods on brand sales and market shares. 5. The storyline of the two programs appeal to different demographic groups. While viewers of both programs may be in the same age range, they likely vary on household size. Viewers of “Soap Opera A” are likely to have younger kids and not a high level of health consciousness (hence more potato chip purchases) whereas the viewers of “Soap Opera B” are likely to have more health conscious singles or couples (hence more purchases of trail mix nuts). The management should investigate the extent to which these consumption pattern differences are indicative of other lifestyle differences (such as differences in health consciousness) and disposable income in order to better target their advertising campaigns. 6. Wash O'Well has decided to focus on its point of sale activities as means to increasing its market share. The options available to Wash O'Well are to go in for standardized data alternatives like Scanner data and diary panels. Before explaining the scanner based audit services, the students should be exposed to how scanner data is collected and what its applications are. In case of markets activated by scanner checkouts, purchases are recorded by passing them over a laser scanner which automatically reads the bar code printed on the package. The cash register is activated, which relates the product code to its price and the amount due is calculated. Information on the newspaper features, displays and the likes are stored and measured with respect to the sales and contribution. The most important benefits of scanner data are accuracy, savings in time and ability to study very short time periods of sales activity. a. Wash O’Well can obtain its detergent sales data from the scanner based audit firm in various levels. It should obtain the data at the individual store level. Wash O'Well should experiment with various shelf spacing, with various kinds of displays and obtain the data. From the data available for various experiments, the management should be able to pinpoint the deficiencies in its point of sale and improve them. Alternatively, Wash O'Well could use the diary panel method to study the purchasing activities. A continuous panel can be used by Wash O'Well. In this method, the panel member records the purchase in a diary and mails the diary at regulated intervals. A change in shelf spacing/ display can be done in one region leaving the other regions unaltered and the results compared. The advantage of such a continuous panel is its accuracy. b. The advantages of a diary panel for Wash O'Well are: ● Provides aggregate sales activity, brand shares and shifts in buyer characteristics and types of retail outlets from month to month. The ability to measure changes in the behavior of individuals, is of real advantage to Wash O'Well as it requires such information. ● Profile of the heavy buyers ● Brand switching and the evidence of stable buying ● Repeat purchase rates for Wash O’Well detergents However, the diary panels have their own limitations: ● Lack of representativeness and hence selection bias due to refusal of individuals to participate in the panels. ● Excessive drop-out rates ● Testing effects wherein heavy purchase activity is noticed because of the novelty of reporting responsibility. The advantages of scanner based audit services are: ● High levels of accuracy ● Time saving ● Ability to study very short time periods of sales activity ● Availability of data at any level, in any combination of stores Students should be encouraged to come out with more data source alternatives for this problem. It should be emphasized that their recommendation on the most optimal data source should be made after going through the pros and cons of each data source. 7a. The two types of consumer purchase panels are the home audit and the mail diary method. The home audit approach is wherein the panel member permits an auditor to check the household stocks of certain product categories at regular intervals. All the used cartons, wrappers are all saved up for recording by the auditor. The other method is the use of a mail diary in which the panel member records the details of each purchase in certain categories and returns the diary at regular intervals. b. The mail diary method is very prevalent in the U.S. c. The advantages and disadvantages of using a consumer panel is discussed in Question 6 (b). PART II SECTION A TEACHING NOTES FOR CHAPTERS CHAPTER SEVEN MARKETING RESEARCH ON THE INTERNET Learning Objectives • Discuss past and current developments of the Internet. • Describe the components of the Internet. • Discuss current commercial research applications. • Discuss the current use of the Internet as a marketing research tool. • Discuss the outlook of future developments. Teaching Suggestions Though there might not be a wide variation among the students regarding familiarity with Internet, it is useful to ask the students about their Internet experience and how much they have researched using this medium. To test their navigation skills, students may be asked to visit an Internet site for collecting information on Secondary data. Here, accessing "census.gov" or “CIA factbook” could be a useful tool. Alternatively, the students may search for industry-specific information such as healthcare or ice cream industry. Data on market size, trends, major players, etc. can be obtained. This chapter can only be taught by assigning students at least two Internet related projects. Students can enhance their learning by using the available resources themselves. Google Analytics is fairly new and a short assignment on that would be useful. It is important for students to understand that although traditional forms of research will continue, the Internet is rapidly gaining momentum in marketing research. Depending on the availability of time, the number of assignments on the Internet could be varied. Students can be asked to develop a questionnaire to survey product preferences and send it via e-mail through the creation of a distribution list. This distribution list should be the e-mail list of friends or classmates. Also, the questionnaire could be posted on the web and fellow students may be asked to access the site and fill out the questionnaire. Websites such as surveymonkey.com and qualtrics.com allow designing of online questionnaires as well as preliminary data analysis for free to a limited extent. Collection of primary information over the Internet has also gained popularity and students should be encouraged to find primary information for a specific industry or market. The main forte of the Internet is probably its advantage in researching secondary information. The Internet competes with several other on-line resources, like professional databases (e.g., Lexis-Nexis, Factiva) to satisfy the information needs of businesses and consumers. Since finding out information about competitor activities is an important task for businesses, students should be encouraged to find out information about competitors within the same industry. Although it may seem relatively easy to find information on the Internet, finding relevant information may not be so easy. Students should be told about the pitfalls and mistakes that occur when searching for information online. The use of the internet for marketing research today should be discussed at length as well as creating familiarity with the latest developments and technologies available to facilitate the online researcher. It is also important that extensive use of the web should also be accompanied by an awareness of traps like identity theft, as well as other piracy and privacy issues. Suggestions for Questions and Problems The number of travel-related sites is constantly increasing. Two prominent examples that can be used for fare searching are http://www.expedia.com and http://www.travelocity.com. In the appropriate menu, the student can enter all the relevant parameters (departure city, arrival city, number of persons, day, time, preferred airline, etc.) and the site retrieves the information immediately for no charge. A very simple approach would be to begin with www.yosemite.com. Another approach would be a travel site (expedia.com), search for California and then for Yosemite. A third approach would be an intuitive Google search that leads to the National Park Service listing for Yosemite National Park https://www.nps.gov/yose/index.htm Design the survey manually or using any of the free survey websites available. Secondary data is one of the main fortes of the Internet. The first step would be to identify the specific countries that are being considered for expansion, and look up the websites of their agricultural departments that can provide information on national, state and district-level agricultural details. These websites can also potentially provide insights on fertilizer usage, crop patterns, yield statistics, future predictions about crops and agricultural practices, and government regulations. Online archives of news articles and current affairs in those countries can provide information on agriculture related affairs in the past years. There could also be insights that can be garnered from consulting reports and national level studies. The web addresses of major US companies are very easy to identify. Once a company is identified, students can find the website via a simple Google search. By surfing the website, students can collect information about what the company does, where its offices or outlets are situated, what services it provides, etc. PART II SECTION A TEACHING NOTES FOR CHAPTERS CHAPTER EIGHT INFORMATION COLLECTION: QUALITATIVE AND OBSERVATIONAL METHODS Learning Objectives • Discuss the need for qualitative research. • Describe the different types of qualitative research methods. • Discuss in‐depth interviews, focus group, and projective techniques in detail. • Discuss various observational methods. • Describe recent applications of qualitative and observational methods. Teaching Suggestions Although qualitative and observational methods are treated together they frequently represent the two poles of a continuum of data collection methods: The contrasting of these methods is a useful way to begin the discussion of primary data collection, which is the concern of the rest of this section of the book. Of course, both are useful exploratory methods and observations should be considered very early in the research design process as either a substitute or a complement to survey methods. Two important points to develop with this chapter are: first the absolute necessity to precede the development of any survey instrument with suitable exploratory work to gain familiarity with the problem, and second, the value of having, multiple measures of the same phenomena using maximally different methods, The errors inherent in observations are very different from those in surveys, so the two methods are usually very complementary, Also, the survey provides the diagnostic insights necessary to understand the observed behavior which otherwise might be quite inexplicable. This chapter lends itself to various interesting field projects which can enhance students’ understanding. For example, students can be sent to observe purchase behavior in a specific section of a grocery store (and pool their experiences in the classroom), or use a contrived observation method to study the quality of services in a local bank or the effects of price bargaining strategies on the quoted price for a major appliance. Instructors could discuss the importance and the value of in-depth telephonic interviews and the situations that warrant a telephonic interview. The instructor should stress the essential requirements of telephonic interviews such as employing an experienced interviewer, providing the interviewer with all relevant information about the organization being interviewed, ensuring that the interviewer creates a conducive environment during the interview for free information flow and ensuring that the interviewer does not spell out his personal opinions or judgments on the subject of the interview. Questions and Problems 1. Non-direct and semi-structured individual interviews are quite different in practice, although they have several overlapping features. A nondirective interview is basically an exploratory research tool. The samples are small, as the usual rule is that interviews will proceed until the interviewer/analyst gets no new insights from the additional interviews. There is no attempt to get a representative sample, instead the objective is to get as wide a set of viewpoints as is possible. A semi-structured interview seeks a representative and substantial sample of the population, which is usually both specialized and difficult to reach. The objectives and information needs are specified quite fully in advance as a result of previous exploratory work. However, the open end question format with potential for probes of interesting responses is more flexible than the average structured survey. This flexibility is dictated both by the complexity of the topic area, and the necessity to communicate with busy people who tend to be impatient with rigidly defined response categories. Ultimately, the objective of a semi-structured interview is to obtain a representative range of answers to specific questions, and not just to explore the topic. 2. The issue posed by this question is one of the most serious problems that affects the Relationship of researchers and managers; namely, the tendency of managers to dispense with subsequent quantification if the exploratory work confirms their initial expectations. Focus group studies are especially prone to this abuse. In this situation, the researcher has a variety of possible responses: (a) The principal argument should be that the objectives of the two focus group studies were to learn how the features would fit into the lifestyles and needs of prospective buyers; specifically how would the features be perceived, what benefits would be foreseen and what problems are anticipated in the pursuit of these perceptions? The discussion leader is bound to elicit lots of enthusiastic comments. This gives the manager plenty of data if he selectively hears only what he wants to hear and downplays the negatives. (b) The environment of a focus group is subject to many distortions. There are group dynamics, which may result in one person who is very enthusiastic dominating and overcoming the reservations of the rest of the group. These factors interact with the usual problems of concept tests, including susceptibility to courtesy bias and inability to incorporate price considerations. (c) Finally, the sample is very non-representative. The people who come to focus groups are quite likely to have more than average interest in new products, services and experiences. 3. The appropriate approach here is the contrived observation method. One possible design is to recruit teams of men and women interviewers to pose as husband and wife during a shopping trip for a specific major appliance. The teams should be matched as closely as possible on all dimensions except ethnicity (that is, be similar in terms of age, dress, education, income, occupation and so forth). Each team should visit as many retailers as possible and negotiate the final price to be paid for the item (such as a refrigerator of certain size and quality with average features and energy consumption). To keep things simple, it would be desirable not to introduce the issue of a trade-in, since these are difficult to appraise. However, credit terms should be sought. Naturally, each pair of shoppers will obtain a range of prices that will depend on differences in the retailers (their margins, the brands they offer, etc.), but if all shopper teams visit all retailers then any difference in mean price will be a meaningful test of the existence of discriminatory treatment (as long as the difference is statistically significant). Data should also be obtained on the interaction between shopper team and salesperson. How long did they have to wait for service? How many models were shown? Was there any attempt to offer a discount or seek the basis for a discount? 4. The problem the consumer organization faces is to draw a valid and defensible sample of products, stores and times of work to conduct their survey. The problems with any arbitrary shopping basket of products is that it pertains to no one in particular, for all shoppers have different requirements. Decisions will thus have to be made regarding the specific products of organic vs non-organic foods to compare, package sizes, the treatment of specials, discounts and coupons, and the potential for shoppers to switch brands or types of products. Most vexing of all is the difficulty of obtaining comparable quality levels of products between stores. This is especially difficult for perishable products such as meats and produce which are used by some stores as loss-leaders, and which can vary enormously in consistency of quality and freshness. A related problem is the lack of comparability of private label and generic (no-name) brands with national brands in this context of organic vs non-organic foods. 5. The first step is to generate hypotheses as to why self-reports of consumption should overstate the frequency of brushing teeth. The most obvious reason is that there is a socially desirable response bias operating—people know they should brush their teeth after every meal, and may feel compelled to report this as their routine behavior. Another problem is disparity in usage rates among family members. It could be that the response rate is poorer among younger members of the family who use less tooth paste. Also, some people don’t use commercial toothpaste, but may brush with baking soda. One way to cope with these problems is to conduct a large scale study and ask randomly selected individuals in the household about many routine behaviors during the previous day—including tooth brushing. This would deflect attention from the real purpose of the study. 6a. Before attempting to recommend a suitable technique for the marketing research group, students should be exposed to explanation of focus groups and in-depth personal interview, in detail. Focus group is the process of obtaining possible ideas or solutions to a marketing problem from a group of respondents through discussions. The research purpose and the research objective are well laid out. Then, the research purpose is broken down into a set of research questions. From these questions, the moderator prepares the specific issues to be addressed in the focus group discussions. The moderator tends to move from a general discussion to specific issues. The emphasis is on the results of the interaction when focused on a series of topics introduced by a discussion leader. A typical focus group has five to nine participants and each participant is encouraged to express his or her views on the topic and to elaborate on views expressed by others. Of late, electronic focus groups have started gaining popularity. In an electronic focus group, the participants type out their responses on a computer terminal and the responses are monitored by the moderator. Focus groups are of three types: * Exploratory focus groups are commonly used at the exploratory phase of the market research process to aid in the precise definition of the problem. * Clinical focus groups are wherein the moderator probes under the level of consciousness of the consumer to obtain a person’s true motivation and feelings. * Experiencing focus groups allow a researcher to experience the emotional framework in which the product is being used. Three or four sessions can help a researcher in uncovering relevant material required for the research problem being studied. Individual in-depth interviews are conducted when the subject matter of the interview is explored in a detailed manner. The in-depth interviews could be nondirective or semi-structured. In a nondirective interview, the interviewer attempts to uncover a list of areas by allowing the respondents maximum freedom to respond. It offers the scope to probe in order to clarify and elaborate interesting responses and when the respondent digresses from the topic the moderator brings the discussion back to the area or topic being pursued. In a semi-structured interview, the topic areas to be pursued are structured but there is enough room for the interviewer to pursue unexpected facts or attitudes that might come out during the interview. In our case, the choice narrows down to a nondirective interview with a selected set of students and faculty and exploratory focus groups to uncover the motivation of students to carry weapons to schools. The research study is on a sensitive topic that seeks to uncover the reasons for a problem that could assume huge proportions. In this case, the student should be urged to come out with their solution that could fit in with the researcher’s needs. The discussion can be centered along the following lines: * Whether group interaction may stimulate new thoughts from the respondents? * Is it better to express sensitive thoughts in a noncompetitive environment? * Is it or is it not important to have time to probe deeper into interesting sidelights? * will the subjects talk freely on such an issue in front of other members? * Is it easy to assemble a diverse focus group that would provide insights into the problem being researched? These are some of the questions that have to be analyzed before arriving at the appropriate technique for Church and Associates. b. The moderator should develop from the research purpose a set of relevant questions which ensures that the researcher and the client agree on the specific objectives before the commencement of the study. From these research questions, the moderator prepares a discussion guide which serves as a checklist for the specific issues being covered. The topic in question is sensitive and it requires an effective moderator to discuss their feelings, anxieties and frustrations as well as the depth of their conviction on relevant issues. The moderator should establish a rapport quickly with the participants and address the group in simple language. The moderator should be flexible so as to uncover certain interesting points but should also revert the discussion back on track, in case of digression. The moderator should guide the discussion in such a way that new topics are introduced to make way for exhausted topics. All the group members should be provided with a chance to express themselves on the subject. c. The students should be asked to develop their research purpose and research objectives for this problem. From the research objectives, a set of research questions can be developed from which the interview questions can be developed. It is important to decide on the structure of the interview. Assuming a structured interview, the following questions can be asked. * Have you known anyone who has carried/been carrying arms in the past 3 months? * What is your assessment of the number of students who carry firearms to the class? * What are the reasons for carrying the firearms to school? (List the possible reasons like Peer pressure, Fear of life, Hike in Self esteem etc.,) * Have the teachers noticed this phenomenon and if yes, have they taken any action in this matter? How have the school authorities reacted in case of complaints? * Do the students brandish the weapon(s) only when provoked or even in unprovoked situations? * What is your advice to the head of the institution in handling the situation? 7. The basis of any interview is the use of words. It should not be assumed that words in one language will mean the same in some other language. The choice of an interpreter should be done with care. After translation into a particular language, it is always better to back translate to check for discrepancies in translation. When words and ideas cannot be communicated in another language properly, it creates a special problem in translation. Apart from this, the cultural factors have to be taken into account while interviewing. It may be unethical to question a subordinate on his boss, political views, personal details, etc., these are some of the difficulties, a researcher might face while interviewing internationally. 8. When information is required on the current behavior, observational methods are used. However, the researchers have long neglected the observation of ongoing behavior as they did not appreciate the considerable benefits offered by observational methods. Casual observation is observation on the extrinsic indicators that are readily available whereas a systematic observation involves looking deeper and noting things that are missed by the casual eye. However, observation becomes the only research alternative in some cases. To obtain insights into search behavior and related issues, direct observational methods are used. In this method, an observer disguised as a shopper watches the shoppers making their purchases. This type of observation can be highly structured. However, it is imperative that the respondents are not aware of the observer. Also, care should be exercised that there is no invasion of privacy. To obtain insights into the various aspects of the underlying beliefs, attitudes and motives of consumers, contrived observation techniques are used. To analyze written material into meaningful units, content analysis is used. Similarly, behavior recording devices have been developed that provide a good alternative to rectifying the deficiencies of human observers. The people meter of A.C. Nielsen to measure the audience’s TV watching behavior, Eye movement recorders, traffic counters are all examples of behavior recording devices. Observation methods have a major limitation in that they cannot observe motives, attitudes or behavior and hence their diagnostic usefulness is reduced. 9. The purpose of the pretest is to match the expectations of the researcher in terms of the information that will be obtained from the questionnaire. The objective of a pretest is to identify the deficiencies in a questionnaire and correct them. The pretests have to be looked at from a macro and a micro level. At the micro level, each question has to be looked at carefully, if they justified their place in the questionnaire. At the macro level, the entire questionnaire should be examined to see for the flow of questions and to check if the questionnaire captured the research objectives. A pretest can identify problems in the questionnaire, which if left uncorrected could lead to erroneous conclusions. Hence, it is always better to pretest as it adds to the enhancement of the quality of the research study. 10a. Qualitative research. Individual interviews conducted at the point of purchase will be a good way of reaching a large number of customers in a short time. All the people purchasing essential oils could be interviewed about when and for what purpose they utilize the product. 10b. Qualitative research. Have a focus group discussion with panel members from both the supporting and opposing populations. The topic of discussion should be what the general public expects from a political party and what changes in the existing agenda would bring in more support. 10c. Observational methods will be very useful in this circumstance. It will help the designers if they know what rides attracts visitors the most. They would also be able to gather by observation the age profile of visitors who are attracted to different rides. This will give them an indication of the enthusiasm with which the new rides will be received by the visitors. To evaluate the amount that people are willing to pay, they can be asked at the point of purchase, how much would they believe the cost of the ride should be. An analysis of the visitors’ responses will help the management understand the price that visitors are willing to pay for each of the rides. 10d. Systematic observation of customers who buy this product should be made at the point of purchase. This will give the company an idea as to which packaging attracts customers the most when the product is placed on the shelves at retail outlets. PART II SECTION B TEACHING NOTES FOR CHAPTERS CHAPTER NINE INFORMATION FROM RESPONDENTS: ISSUES IN DATA COLLECTION Outline of the Chapter • Discuss the different kinds of information that are collected through surveys. • Discuss the errors that occur while conducting a survey. • Discuss in-depth interviews, focus group, and projective techniques in detail. • Discuss the various factors that influence the selection of the survey method to use. • Discuss the ethical issues involved in collecting data from respondents. Teaching Suggestions Many instructors will choose to use this chapter in conjunction with the chapter on questionnaire development. This helps to reinforce the two key points of this section of the course, that: (a) no survey method can simultaneously satisfy all criteria, and (b) that various issues have to be considered before deciding on data collection. The solution to these problems lies in anticipating the problems and sources of errors, and minimizing them during the design and analysis of the study. This may require using combinations of survey methods and taking account of the probable direction of bias during the interpretation of the results. There are several teaching options open to the instructor, depending on the amount of time devoted to data collection methods. One approach is to rely on the cases and discussion questions. A second approach is to expand on the material in the chapter, in one or more of the following areas: (a) Response effects—a supplementary note is provided as a resource, (b) Interviewer recruiting, training and controlling to minimize interviewer error (c) Ethical issues in data collection. The instructor might want to use some of the reader surveys published in magazines on a variety of issues and discuss with the class the “validity” of these types of sampling procedures. Alternatively, a discussion could be organized around key ethical concerns such as the invasion of privacy or use of selling methods disguised as research. The key teaching point of this chapter is that all data collection methods take place in a subjective context and must be tailored to meet the requirements of a particular research situation. Questions and Problems 1. It is very unlikely that the average DIY respondent who has repainted a room has done so within the past week or month. There is likely to be a significant amount of forgetting and/or compression of time. These problems are accentuated by the need to establish the kind of paint, which is perhaps not an important fact to many people who rely on their dealer for a recommendation. To cope with these problems it is necessary to first establish the context of the redecorating job, to get the respondent thinking about the problems (i.e., of surface cleaning, time of year, etc.), and then ask how the type of paint was chosen, before asking the question of interest. Each of these steps will help to refresh the respondent's memory of the specific event. One important caveat in this type of survey is to be sure that the family decision-making process is understood, and the decision-maker is the one being interviewed. It may be that the husband bought the paint and the wife applied it, or vice versa. The person who applies the paint may not know anything about the kind that was used. 2. Ethical considerations arise whenever respondents' rights are abused. There is no right answer to this question and whether students put the emphasis on research quality or on respondent rights depends on their own orientation. Students should be encouraged to think of alternative ways to collect data on some of the situations mentioned in this chapter so that both interview quality and respondent rights are respected. 3. The available evidence suggests that the biasing effects of an interview are more serious in a personal interview. There are several reasons: personal interviews are longer and are more like social interactions and accentuate problems of prestige seeking and social desirability and the physical presence of the interviewer means there are many more non-verbal cues for the respondent to use. In both types of interview, the interviewer can bias the results through selective listening, probing and recording. For the most part careful attention to the basics will minimize the problem. This includes: (a) thorough testing of the question, so the interviewer doesn't need to resort to alternative wording, (b) use of required probes when there is doubt as to whether the interviewer will follow-up a partial answers, (c) thorough training of interviewers so they understand the purpose of each question and the possible responses, and have had an opportunity to rehearse the question during the training session and (d) tight supervision and control, through follow-up validation of completed interviews and monitoring telephone interviews, to quickly correct problems as they occur. 4. Because the majority of commuters or shoppers use cars the survey should emphasize the reasons for driving rather than taking urban transit. The prospective respondents should be asked to participate in a survey of their experiences and problems in getting to their destination. There are two reasons for this approach; first, it is something they are familiar and involved with, so they will be more willing to respond, and secondly, the plight of the transit system is probably best understood from the reasons it is not used. 5. This set of questions will easily stimulate a lively discussion. (a) One overriding issue is the declining cooperation of respondents caused by the overuse of surveys and unpleasant respondent experiences. Taking more time than promised is one way to generate an unfavorable experience. The class could be asked to suggest others. Others could be: ▪ Misrepresenting the compensation ▪ Not sending a promised report of complete results ▪ A rude, unpleasant, or disinterested interviewer ▪ inconvenient timing ▪ Personal questions One survey found that 74% of those who had been interviewed within the last year had a favorable overall impression of their experience. However, 19% thought the questions were too personal, 17% thought the interview was too long (rising to 45% when the interview was over 20 minutes long), and 19% believed the interviewer was not interested in their opinions. The following questions get into the ethics of the profession. (b) This question involves an outright deception and most students will be offended by it. However, others may argue that it really harms no one and will help the survey. 5(c) is more subtle since it was only implied that the group would not be taped. More subtle yet is the common practice to view surreptitiously the groups without informing them. (c) The general issue is the ethics of deception vs. the validity of the research. Such a trade-off is nearly always required. This case may be easily handled by the students. But there are others more subtle. What about the use of a deceptive cover story which is usually needed? For example, an advertising test is conducted on the basis that a program test is the focus. Other kinds of deception are common such as the use of a plant to stimulate discussion in a focus group. 6. The students can be made to look at this problem in many different ways. First, the research purpose and the research objective should be thought out. The research questions should then be formulated from the research objective and thus the information requirements can be assessed. The information required can be collected using focus groups, protocol tasks, in-depth interviews or surveys. The choice of a method depends upon the economics of cost, time and effort. The students should be made to go through an exercise involving the various methods of data collection, identify the pros and cons of each and then make their decision on the data collection method. One way of looking at it would be to identify the aspects of the decision making process that consumers use to choose Brand M butter or the competitors. The premise would be that there are some reasons underlying the consumer purchase behavior that is causing people to switch to/ buy other brands and hence cause loss in sales for Brand M butter. Let us assume that a survey method is used by the product manager to collect the information required from the respondents. It should be kept in mind that effective implementation of the project requires the use of the correct choice of survey method, be it administered through telephone, or mail or by means of a personal interview. However, there are some distinct disadvantages to surveys that stem from the social interaction between the interviewer and the respondent. In identifying the decision making process of the consumers in buying butter, the focus should not be on the decisions made in the past but on the process by which consumers evaluate alternatives. The manager wishes to measure the relationship between actions and needs, desires, preferences, motives and goals. The product manager would prefer to get information on the following counts: * Whether the respondents bought brand M butter or not? * If they did buy brand M butter, where did they usually buy it and when? Will they continue to do so in the near future? * How often did they buy brand M? * Do they persist with brand M or do they opt for other brands too? If they do opt for other brands at times, why do they switch? * If they don't buy brand M, have they ever tried using brand M? What is their source of dissatisfaction with brand M? * If they have never tried brand M, what are the attributes of their favorite brand of butter? Responses to the above mentioned outline will give the manager a fair idea of the key aspects of the decision making process of the consumers in buying brand M. 7. Observational method is limited to obtaining information on current behavior. Observation can be used as an important exploratory method. For instance, a customer service manager of a bank can monitor the customer lines in front of the teller service counter and identify potential problems. If the waiting time is too long, it may lead to customer dissatisfaction and the manager would do well to have an extra hand to help out in times of need. Alternatively, if the lines are too long always, then he or she may decide to hire another person to do the job. Also, observation may be the cheapest way to collect purely behavioral data. It may sometimes prove to be the only research alternative especially in doing research with young children who cannot articulate their preferences or motives. Observation methods are costly and time consuming and cannot observe motives, attitudes or usefulness and hence diagnostic usefulness is reduced. These cognitive factors are manifested in the observed behavior but there are so many other confounding factors and hence it is difficult to pinpoint the exact factor. Very little control can be exercised over the sample. Questioning, on the other hand, is a more complete way of obtaining the motives and preferences from the respondents. There is a social interaction between the respondent and the interviewer and hence the chances of clarifying for further details or developing on a line of questioning can be useful. This is especially true in case of sensitive subjects, though there is a school of thought that believes that respondents may feel uncomfortable while answering sensitive questions as they may not admit a negatively valued characteristic to an interviewer. Also, the accuracy and the quality of information received from the respondents is also questionable, as there is little room for the respondent to consult their records or interact with their family members. There is greater degree of control obtained with a sample as compared to observational methods. 8a. The choice of a data collection method is a critical point in the research process. The three basic survey methods are a. Personal interview b. Mail survey c. Telephone interview A brief outline of the various factors to be considered while choosing a survey method is given. ● Type of population: The reading and writing skills of the population and their motivation to cooperate are to be considered in choosing the mode of data collection. If the workers are not well educated and whose reading and writing skills are less than ordinary, administering a questionnaire through a personal interview would be a better proposition as compared to a telephone survey or a mail survey. Alternatively, if the population meets at a particular place, administering questionnaires through groups is a good possibility. ● Sampling: The factory management roster will give the precise addresses and telephone numbers of all the workers and hence using a mail survey or a telephone interview is possible. However, respondent designation, i.e. the worker only should be the respondent, is a major issue and hence mail survey is not appropriate as the researcher has no control over the issue. ● Question form: if the questionnaire involves some visual cues or pictures, mail survey and telephonic interviews are out of question. ● Response rates: Mail response rate typically tend to be low. Similarly telephone surveys draw a lower response as compared to personal interview. United Machine tools can administer the questionnaire in groups and hence obtain a good response rate. ● Cost: The mail survey and telephone surveys tend to cost less than the personal interviews. However, if the questionnaire is administered in groups, then the cost of administering it in person is reduced further. The budget allocated by United machine tools is a factor in determining the mode of the survey method. ● Time: The time involved in a mail survey is higher as compared to a telephone survey or personal survey. However, the workers congregate at the factory daily and hence collecting the required sample is easier. b. The researcher would like to have close ended questions because coding is easier and therefore data analysis is easier. If the questionnaire had open ended questions and was self administered, then the responses will not be comparable across respondents and hence coding becomes difficult. c. Employee surveys help companies look inward for self improvement. In an organization, it is always possible that the top management lose sight of what happens in the lower levels. The job satisfaction survey will help the company to identify its strengths and weaknesses and helps the employees feel a part of the larger mechanism. The broad areas that could be covered in a job satisfaction survey amongst the United machine tools workers would center around the following. The specific questions are not discussed here. Working conditions- * Workers' satisfaction levels towards statutory and other welfare schemes. * Working Conditions * Workers' attitudes to their supervisor * Satisfaction with remuneration and benefits; career advancement and awards * Workers' attitudes towards fellow workmen * Company's interest in the employee welfare and commitment to good life of employees * Workers' attitudes towards job security and intrinsic job value d. The possible sources of survey error in this study are: * Sample tested is not representative of the population * The workers selected to interview are uncooperative, unwilling to participate in the survey or do not possess the knowledge required to respond to the survey. * Interviewer does not administer the questionnaire properly. * The interviewer does not understand the responses and records the responses incorrectly. e. It should be highlighted that the students should not assume that the conditions in the rest of the world are not the same as the US and hence the researcher should adapt himself or herself to the local situation. A few pointers in this regard are explained below: While choosing the survey method, Houston resources consultancy should take into account the economic, cultural and social factors in India. While the factors discussed in (a) are important, Houston resources consultancy should assess whether it has the ability to conduct a survey in India or should it collaborate with an Indian agency to do the job. The language the questionnaire should be administered in should be ascertained. The questionnaire should be translated into the local language and then back translated into English to ensure that the translation is correct Also, while considering the choice of the survey method, it should be borne in mind that telephone ownership in India is not as prevalent as in the US and hence the telephone survey is not a viable proposition. 9a. It is not easy to categorize that a seven-page questionnaire will draw a good response or a poor response. In case of a mail survey or a telephone survey, the length of a questionnaire will definitely affect the response rate. It should be borne in mind that longer questionnaires are better administered through personal interviews as opposed to mail and telephone surveys. It is because the interviewer establishes a rapport with the respondent and hence refusing to cooperate becomes difficult. In case of a questionnaire administered to busy executives through personal interviews, the shorter the questionnaire the better. In our case, we are looking at a questionnaire to be completed by the customers at surf stores. It should be remembered that as accuracy of the responses diminishes with increase in questionnaire length, owing to reasons like fatigue or anxiety to get it over with and this leads to response bias. Therefore, Hirst should be prepared for a poor response rate and for incomplete information on the questionnaires. b. Hirst cannot be sure that the individuals buying at surf stores are users themselves. Such people may not be capable of answering his questions. Also, the sample thus chosen at the stores may not be representative of the population. This leads to selection bias as the sample is not representative of the population defined. Also due to the length of the questionnaire, some of the positive motivations that the respondent had to begin with could have dissipated leading to response bias and distortion of responses. 10 a. “Families for the future” should choose a random, representative sample for its survey. The refusal or non response rate can lead to a major source of error. In case of responses received, respondents could have been incapable of responding and hence this leads to response bias. The respondents might distort their responses to enhance their prestige in the eyes of the researcher and this leads to Social desirability response bias. b. In order to achieve a fair response rate, some sort of an incentive or honorarium can be kept in the survey for the participants. This incentive can increase the survey responses to a certain extent. Also, a reminder can be sent after mailing the survey to all non-respondents so as to elicit their responses. In case the response rates are poor even after the mailing of the reminder, fresh surveys can be mailed. The questionnaire has to be kept short in order that the respondents don't get fatigued and are as accurate as possible in their responses. SUPPLEMENTARY NOTES ON RESPONSE EFFECTS IN SURVEYS (Prepared by Al Silk, MIT) Sudman and Bradburn have developed a comprehensive review of available empirical studies bearing on the sources and magnitude of “response effects” in survey research.l A response effect is a systematic error or bias in an aggregate or average value of some measurement caused by a distortion factor present in the measurement process. An example of such an effect would be underreporting of purchasing of some products due to forgetting. Conceptually then: where: O, T. and εi are sample aggregates or means. The authors assume that response errors are independent of true values. Method A secondary analysis was carried out on virtually all published studies response effects—the raw materials are found in a bibliography of 935 articles.Each article was scrutinized for information reported relating to the size of response effect attributable to some particular procedure or factor. The following measures of the relative magnitude of response effect report in any study was computed: where S is the standard deviation of the population, obtained from the validation information if possible. If data on S were not reported, RE was computed from: For studies reporting information about attitudes (rather than behavior), the weighted mean of all observations was used for validation. Suppose, for example, attitudes on some topic had been measured via telephone interviews for one sample and by means of personal interviews with the second equivalent sample. Let Xl and X2 be the mean attitude scores for the two groups and G the grand mean of the two samples combined. Then: where Sg is the standard derivation computed from the grand mean. Model of Response Effects for interviews The various sources of response effect that had been studied were grouped as independent variables according to the conceptual model of an interview shown in Figure 1. Mean values of RE were computed for the various estimates of a particular type or source of response effect reported in the studies found in the literature. Three conceptually distinct sources of response effects were identified by grouping forty-six specific variables or types of effects: (a) variables that derive from the nature and structure of the task. (b) variables that derive from the characteristics of interviewers, (c) variables that derive from characteristics of respondents. The task variables were further divided into three sub classes: (i) variables relating to the structure of the task and method of administration, (ii) variables relating to problems of self-presentation on the part of the respondent. (iii) variables relating to the saliency of the task to the respondent. The following hypotheses were formulated: 1. The greater the degree of structure, the lower the relative response effects will be. 2. The greater the problems of self-presentation evoked by a question (i.e., the more socially desirable some of the answers are), the greater the pressure on the respondent to answer a question or the more controversial a question, the higher the relative response effects will be. 3. The greater the saliency of the information required, the lower the relative response effects will be. a) The more recent the event, the more important it is to the respondent, the less complex, and the more positive affect toward the event, the lower the relative response effects will be. b) The more recent the event, the less complex, the more positive affect, and the dearer the subject of a question about an attitude, the lower the relative response effects will be. 4. The greater the degree of structure in the interviewer’s role, the lower the relative response effects will be. 5. The greater the degree to which the interviewer actually carries out the role demands, the lower the relative response effects will be. 6. The greater the saliency of an interviewer's extra-role characteristic for the question being asked, the greater the relative response effects will be. 7. The better motivated the respondent to perform his task of providing information, the less the relative response effects will be. Summary of Results 1. Non-Threatening Behavioral Questions (a) Task Variables (i) memory Actors most important (mean RE = .15) (ii) other task variables of some importance, averaging about RE = .05. (b) interviewer and demographic characteristics are of little or no importance with RE ≅ 0. 2. Threatening Behavioral Questions Response effects are generally larger for threatening than non-threatening behavioral questions. (a) task variables, average RE = .24 (b) memory, average RE = .18 (c) respondent - interviewer variables least important average RE ≅ 0, except for college students and studies of sexual behavior. 3. Attitudes Task variables are more important causes of response effects in attitudinal studies than are respondent—interviewer characteristics, except where these characteristics are highly related to the attitudes being measured (e.g., studies of racial or sex-role attitudes). Instructor Manual for Marketing Research V. Kumar; Robert P. Leone; David A. Aaker; George S. Day 9781119497493