This document contains Chapters 6 to 7 CHAPTER 6 PERSONNEL SELECTION Objectives After reading this chapter, you should be able to Understand the concepts of reliability, validity, and utility. Understand the validity evidence for various selection methods. Discuss approaches to the more effective use for application blanks, reference checks, biographical data, testing, and various other selection methods programs in order to increase the validity and legal defensibility of each. Discuss the approaches available for drug testing. Describe the validity of different approaches to interviewing. Explain how the various types of candidate information should be integrated and evaluated. CHAPTER 6 - SUMMARY I. Overview. Personnel selection and retention is key to organizational effectiveness The use of validated selection criteria is a HPWP Matching model of selection predicts employee effectiveness Selection includes applications, reference checks, testing, and interviews Selection is the process of gathering and assessing information about job candidates to make personnel decisions Steps in the development and evaluation of a selection procedure 1. Perform job analysis and HR planning 2. Develop recruitment strategy/Develop selection procedures 3. Determine validity of selection methods 4. Determine weighting system for data from selection methods II. Selection Methods: Are They Effective? A. What Is Reliability? 1. Reliability is the consistency of measurement 2. Reliable methods should have a correlation coefficient of .8 or greater. 3. Goal should be to reduce error in measurement as much as possible and achieve high reliability Reliable methods (e.g., tests, interviews) tend to be long Reliability is a necessary but insufficient condition for validity B. What Is Validity? Selection method is considered valid if it predicts some important criterion, close in meaning to job relatedness Extent to which a method achieves the aims of the user Examples of criteria of interest: performance, turnover, employee theft Empirical or criterion-related validity is a statistical relationship between scores on a predictor (selection method) and a criterion (e.g., job performance). a) Relationship is reported as a correlation coefficient from -.1 to +1.0 b) Higher correlations indicate stronger validity Higher correlations typically result in greater utility or economic payoff Sample size can impact statistical significance of results Conceptual match is also important for instance, cognitively loaded predictors predict task performance Content validity assesses extent to which selection method represent the actual requirements of the job Asking a job applicant to perform spreadsheet activities when you are hiring someone to work with spreadsheets-Spreadsheet activity has “content validity” Passing accuracy test at the NFL combine has “content validity” for quarterbacks Validity generalization invokes evidence from past studies of a selection method using meta-analysis. Results may then be applied to a new and similar job and/or setting. Appropriate when sample size is too small C. What Is Utility? Utility is economic gain (dollar savings or profits) from using a particular selection method Methods with high validity, low cost and considering the number of applicants and positions have higher utility The basic formula: estimates the increase in revenue as a function of the use of the selection method after subtracting the cost of the method High utility requires low selection ratios and thus is also related to the ability of the organization to attract a large number of qualified applicants for each position they need to fill Selection methods with high validity but that have lower costs have more utility (more cost effective) III. Application Blanks and Biographical Data A. Both fall under the same legal guidelines as tests; must be careful to not disproportionately screen out protected classes and not ask illegal questions B. Best way to use and interpret information on an application blank is to use a weighting scoring system 1. Weights are derived from empirical validation research C. A Discrepancy between Research and Practice: Use Weighted Application Blanks and Biographical Data. 1. Weighted application blanks (WABs), a) Rarely used by decision makers b) Derive an objective weighting system based on an empirical study c) Improves validity, particularly in predicting future turnover 2. Biographical information blanks (BIBs) a) More personal and experiential (based on personal background and life experiences) Tool in predicting job turnover, job choice, and job performance Used in the military and insurance industry (average validity is .35) 3. Accomplishment records a) Limited to explanations of accomplishments b) Accomplishments should be compatible with challenges facing the organization c) The applicant’s past performance is more valid than educational criteria in assessing managerial potential d) Particularly valid/effective for managerial and professional jobs D. How Do You Derive WAB or BIB or Accomplishment Record Weights? 1. Should have large sample size (100). 2. Correlate responses to performance or turnover data. 3. Weights determined by items on which ineffective and effective employees differ and magnitude of the relationship of the item with the job. 4. Research strongly supports the use of WABs, BIBs and accomplishment record in selection (more so than using the application blank informally). 5. Requires sufficient data and research expertise to be successful IV. Reference Checks and Background Checks Companies are now “Googling” applicants names and searching social networking sites Fear of lawsuits and claims of negligent hiring To assess the potential success of the applicant for the job a) Former employers reluctant to provide information about former employees for fear of lawsuits b) Companies have been sued for not being honest regarding former employees B. Used to verify information since 20-25% of applicants include at least one fabrication C. What is the Validity of Reference Checks? 1. Letters of reference are almost always positive 2. Generally low validity in the .20-.30 range for reference checks as predictors of job performance 3. Constructed “letters of reference” are more effective Letters of reference constructed as appraised forms that evaluators use to determine the candidates’ KASOCs or accomplishments 4. Good faith efforts to obtain verification of employment history make it possible for a company to defend against negligent hiring lawsuits D. What are the Legal Implications of Doing Background Checks on Job Candidates? 1. Background checks should include criminal records only if applicable to job 2. Employers should try to obtain information to avoid potential negligent hiring lawsuits 3. Fair Credit in Reporting Act (FCRA), amended in 2005, and state law both have restrictions on what makes an “investigative consumer report” legal Notice in writing to job candidate being investigated Summary of rights under federal law to job candidate Certify to investigating company that you will comply with federal and state law Copy of the report in a letter to the person being investigated 4. Consult with the National Association of Professional Background Screeners regarding firms to use for background and reference checks V. Personnel Testing A. Corporate America uses General Mental Ability (GMA)/ cognitive ability tests in addition to various forms of personality or motivational testing to screen applicants What Is a Cognitive or General Mental Ability Test? Measures aptitude or general mental ability or capacity to acquire knowledge Used to predict future performance Achievement tests measure effects of knowledge obtained in standardized format (e.g. a final exam) Knowledge-based tests assess sample requirements of job Easier to defend in terms of job relatedness Validity of .48 with job performance Expensive to develop Wonderlic Personnel Test is one of the most widely used of the GMA tests a) Data from 1938 for over 3 million applicants b) 50 questions covering mathematics, vocabulary, spatial relations, perceptual speed, and analogies c) Used by the NFL as part of pre-draft assessment 5. GMA tests are ideal for jobs if considerable learning or training on the job is required and where a more ‘job-related” knowledge-based test is inappropriate or unavailable 6. Controversial because they can result in adverse impact and thus create legal problem 7. The relationship between test scores and performance increase as training and jobs become more cognitively complex and mentally challenging 8. Research supports validity and predictive value of GMA tests (Figure 6-4 p. 198) C. What Are Tests of Specific Mental Ability? 1. Bennett Mechanical Comprehension Test is used for mechanical ability The Minnesota Clerical Test is popular for measuring clerical ability Research favors the use of GMA tests over specific abilities D. Are There Racial Differences in Test Performance? The Griggs and Albemarle cases illustrate the case for disparate impact and job relatedness African Americans score lower than whites on cognitive ability tests Hispanics fall midway between the average black and white scores Questions remain regarding validity generalization results for cognitive ability tests Research indicates a combination of cognitive and non-cognitive assessment may be a better predictor of individual performance than GMA by itself a) Such an approach can also reduce the level of adverse impact b) Validity is increased with this approach E. Why Do Minorities Score Lower Than Whites on Cognitive Ability Tests? 1. There is no clear answer 2. Differences are not created by the tests 3. Related to inferior educational experiences 4. Require further information related to the development of cognitive abilities How Do Organizations Deal with Race Differences on Cognitive Ability Tests? Decisions based solely on test scores are likely to result in adverse impact One strategy is to set a minimum cutoff score so that there is no violation of the 80% rule This approach results in lost utility GMA has a linear relationship with performance, therefore lowering the cut off score could result in the hiring of people who are less qualified Banding – group test scores based on data indicating that the bands of scores are not significantly different from one another Banding only has big effect on adverse impact when minority preference within a band is used for selection This approach may be illegal Remove “culturally biased” questions on GMA tests a) Research does not support this recommendation 5. Research suggests the following: Targeted recruitment Focus on multiple aspects of job performance Augment the GMA test with non-cognitive methods Use tests with greater adverse impact later in the selection process Use methods with content validity such as work samples G. What are Physical or Psychomotor Test? Physical tests are classification of tests designed to assess physical attributes such as endurance and balance Strong criterion related validity for predicting accidents and injury Physical ability test may have adverse impact against women Sensory ability focus on hearing/sight/reaction time, test valid for prediction of vigilant behavior H. What Is Personality/Motivational/Dispositional Testing? 1. Performance is a function of both ability and motivation 2. Use of personality tests intended to measure personality and motivation is increasing 3. 35% OF US companies use personality testing in selection 4. There is increasing evidence that these methods provide a valid means to predict counterproductive work behavior 5. The transition to team based work, and the need to predict how people behave and perform in groups could be driving the increase in usage 6. What Is Personality? a) An individual's consistent pattern of behavior (psychological traits) b) Many researchers use a five-factor model (FFM/ Big Five) of personality i) Neuroticism/Emotional stability ii) Extraversion/ Introversion iii) Openness to experience iv) Agreeableness v) Conscientiousness c) Emotional Intelligence i) Multidimensional form or subset of social intelligence or a form of social literacy ii) Criticism due to differences in definitions of the construct and the claims of validity and incremental validity d) Core Self-Evaluations Broad and general personality trait composed of 4 heavily researched traits self-esteem self-efficacy Neuroticism Locus of control e) Research indicates useful incremental validity for these measures beyond the Big Five and other selection models or tools 7. How Do We Measure Personality? a) Projective tests disguise the purpose and scoring process from the test taker i) Rorschach Inkblot Test Miner Sentence Completion Scale (MSCS) specifically designed for use in employment, uses 7 areas to measure motivation to manage others Thematic Apperception Test is valid for predicting managerial and entrepreneurial success by assessing the test taker’s need for power, affiliation, and achievement Graphology or handwriting analysis, no evidence of validity 8. Self-Report Personality Inventories: purpose to measure personality or motivation with the respondent knowing the purpose and/or scoring procedure of the test Minnesota Multiphasic Personality Inventory (MMPI) identifies pathological problems can be used to defend against negligent hiring claims 16 Personality Factors (16PF) used to screen for counterproductive behavior NEO Personality Inventory is the most reliable and valid of FFM measures Myers-Briggs Type Indicator (MBTI) not useful for selection I. What Is the Validity of Personality Tests? 1. The validity evidence is mixed a) Some have adequate validity b) MSCS has average validity of .35 for selecting managers, no peer review of research Research on FFM found Conscientiousness (C) and emotional stability (ES) have validity for all jobs with Conscientiousness the most valid of the FFM (.31) The other 4 factors are valid for certain situations Extraversion correlated to jobs with a strong social component e.g. sales and managerial positions High scores on Openness to Experience related to training acceptance A combination of FFM factors can more successfully predict important criteria than factors in isolation 2. Use of FFM sub-factors increases validity 3. Why is the validity relatively low? Applicants can fake on self-report instruments Poor study designs and small samples used in the research The relationship between personality and performance is curvilinear Behavior is situationally determined; context and frame of reference can help J. Approaches to the Prediction of Particular Criteria 1. Predicting (and reducing) Voluntary Turnover a) Rely on employee referrals b) Put weight on tenure in previous jobs c) Measure intent to quit d) Measure the applicant’s desire/motivations for the position e) Use disguised-purpose dispositional measures Job Compatibility Questionnaire (JCQ) used to predict job tenure and turnover with strong validity evidence for many low-skilled jobs FFM and EI add incremental validity to the prediction of turnover with managerial positions No evidence of adverse impact has been found. 2. Can We Predict Employee Theft? a) Honesty/Integrity Tests i) Popular since the Employee Polygraph Protection Act (1998) banned the use of polygraph for selection ii) Designed to measure attitudes toward theft and may include questions concerning beliefs, judgments, and perceptions on ease of theft Validity evidence for integrity tests is fairly strong (.41) with no adverse impact Concern that most validity studies were conducted by the test publishers Few criterion-related validity studies use theft as the criterion These test have also been shown to predict job performance in general Can We Identify Applicant Who Will Provide Good Customer Service? Service Orientation Index Can We Identify Bad & Risky (and Costly) Drivers? Previous driving record is single best predictor of on-the-job record There is a link between personality traits and risky driving behavior Safety Locus of Control Scale was developed to predict likelihood of accidents K. How Do You Establish a Testing Program? 1. Study the test manual carefully. 2. Check reviews of the test in the Mental Measurements Yearbook (see http://buros.unl.edu/buros/jsp/search.jsp) 3. Get names of companies using the test a) Call a sample; conducted any adverse impact and validity studies b) Find out if it has had any legal actions (adverse impact) 4. Obtain a copy of the test and examine all items VI. Drug Testing A. Drug abuse has been linked to employee theft, accidents, absences, and other counterproductive behavior B. 87% of firms do drug testing C. Most common type of test is a clinical test of urine samples (immunoassay) D. Hair analysis is more reliable and less invasive E. Tests give little information on the amount of drug used, when it was used, how frequently, and the applicant's effectiveness on the job F. Drug testing is legal for preemployment screening and on-the-job assessment G. State laws and collective bargaining agreements apply H. Drug testing has been challenged under privacy provisions of state constitutions I. Is Some Testing an Invasion of Privacy? Guidelines and standards protect the confidentiality of test results Interpersonal interactions exchange information and thus violate individuals’ privacy to some degree In the interests of high productivity and staying within the law, they assert organization may need to violate an individual’s privacy to a certain extent VII. Performance Testing/Work Samples Performance tests involve job behaviors or tasks required for the job; measure KASOC’s or competencies Performance testing involves actual “doing” rather than “knowing how” A work sample test reflects actual job responsibilities and tasks without the risk Conducted under controlled circumstances Tailored to match important activities of the job Simulations 1. Duplicate real-life event but eliminate the risks of danger or damage Situational Judgment Tests (SJT) Number of job-related situations presented in written, verbal, or visual (video) form Incremental validity above other forms of testing The performance testing process should be Standardized Consistent in administration Include specific standards of performance Proctored web-based testing for SJTs, biodata, and personality testing has benefits above paper-and-pencil testing What Is an Assessment Center? Enables standardized evaluation of behavior on multiple inputs Multiple trained observers/assessors and exercises are used Used most frequently for managerial and supervisory selection Individual's strengths and weaknesses assessed through performance tests Allows for direct comparison of internal and external candidates Assessment of job dimensions and competencies from job analysis a) Dimensions are clusters of behavior that are: i) Specific ii) Observable iii) Verifiable iv) Can be reliably and logically classified together b) Assessment center dimensions (Figure 6-7 p. 216) i) Leadership ii) Interpersonal iii) Organizing and planning iv) Perception and analysis v) Decision making vi) Oral and nonverbal communication vii) Adaptability viii) Decisiveness ix) Written communications Common assessment center exercises a) In - Basket b) Leaderless group discussions c) Oral presentation d) Role playing H. How Are Assessments Done? All candidates perform the same tasks Candidates observed by different trained assessors in each exercise Assessors meet to rate each candidate to derive and overall assessment rating Evidence that assessment can be broken down to reduce cost, and that testing can be completed online with essentially the same results I. What are the Validity and Adverse Impact of Assessment Centers and Other Performance Tests? 1. Validity a) High validity for managerial positions b) Proven validity for law enforcement personnel 2. Lower adverse impact than GMA 3. More defendable in court 4. Method is ideal when there are both internal and external candidates J. Performance Appraisals/Competency Assessment 1. Little research on predictive capabilities of performance 360 competency ratings and assessment center data better prediction and less adverse impact than supervisor ratings of potential VIII. Interviews A. Most common personnel selection tool B. Typically used as the last "test" in the selection hurdle C. Typically not conducted in a high validity manner, however, when properly conducted can be quite valid D. What Factors Affect the Employment Interview? 1. Attributes of the applicant can cause rating bias (stereotyping) 2. Attributes of the interviewer can cause rating bias (e.g. similar-to-me) 3. Situational attributes impacting attention to information can cause rating bias E. What Is the Validity of Employment Interviews? Validity is hampered by factors and biases but can be quite high Unstructured interviews have a validity of .31 with job performance Structured interviews have a validity of .43 with job performance 2. Recommendations for “high validity” interview: a) Derive questions from job analysis results b) Using structured interviews (telephone or live) c) Using consistent rating standards d) Averaging across 3 or more independent interviewers will enhance validity of unstructured interviews 3. Interview is a “test” and subject to same EEO laws Process is similar to subjective nature of the performance appraisal, need to avoid “excessive subjectivity” Subject to theories based on intent and impact Sex Discrimination has mixed findings, often dependent on the job Race Discrimination has mixed evidence, higher ratings when interviewers and interviewees are the same race Age Discrimination depends upon the job under study Disability Discrimination evidence insufficient to support conclusions F. How Do We Improve the Validity of Interviews? 1. Standardize interview physical environment 2. Comprehensive interviewer training a) Interviewers should Use job related information Reduce rating bias through practice Communicate effectively Use interview guides or outlines Exchange information related to the position 3. Interview Content a) Exclude traits that can be measured thru more valid methods b) Include interpersonal and motivational factors c) Match interview questions with the job analysis data d) Avoid biased language e) Limit the amount of pre-interview information to applicant’s qualifications f) Encourage note taking g) Be aware of candidate impression management behaviors G. What Are Major Types of Interviews? 1. Structured interviews ask the same questions of all candidates 2. Group/panel interviews consist of multiple interviewers 3. Situational interviews require interviewees to describe how they would behave in specific situations 4. Behavioral interviews candidates describe actual experiences they have had in dealing with specific job related issues H. The “Bottom Line” on Interview Validity 1. Behavioral more valid 2. “High validity” interviews should be structured with behavioral questions derived from a job analysis and involving more than one trained interviewer using a structured interview rating form 3. Interview data should not be overemphasized, but appropriately considered with other valid information IX. Combining Data from Various Selection Methods A. Use multiple approaches for selection B. Possible approaches for personnel decisions Clinical or holistic analysis without any formal method of weighting scores Equal weighting of methods with rank ordering of standardized summed scores Weigh scores based on empirical validity of each method; Recommended method Weighting and mathematically combining scores known as actuarial prediction C. Legal implications of using a multiple-step process 1. Conn v. Teal determined that even if the company hires proportionately, all steps in the process need to be job related and free of illegal discrimination 2. Multiple steps increase the validity of the process 3. Helpful for job relatedness claims D. Individual Assessment (IA) Popular approach for selection of managers Based on an overall assessment by psychologists Holistic combination does not hold up to scientific scrutiny Little evidence to support this approach X. Personnel Selection for Overseas Assignments. A. Expatriate failure is most likely caused by poor selection Over 50% of multinational companies had expatriate failure rates of over 20% Failure was cited as being caused by: a) Spouse’s adjustment to new environment b) Manager’s adjustment to new culture c) Manager’s immaturity d) Manager’s inability to cope e) Manager’s lack of technical competence f) Manager’s lack of motivation to work overseas B. Selection of expatriates should focus on relational ability including adaptability and the ability to interact well with a diverse group of people C. Use a decision model to determine the advantage of hiring a host national (Figure 6-12) D. Steps used to optimize expatriate staffing decisions Applicant assessed on skills needed overseas that he/she possesses or can develop in three-month training program Personality traits examined to determine if applicant is suited for assignment E. Weights for Expatriate Selection Potential candidates are identified Promising candidates contacted and given overview of the assignment (RJP) Applicants are examined using several selection methods Global Assignment Preparedness Survey – assesses candidates on six dimensions, including flexibility Cross-Cultural Adaptability Inventory – focuses on ability to adapt to new situations and interact with different people from oneself Candidates are interviewed. Expatriates are recommended as interviewers The more rigorous the selection process, the better the success rate XI. Selection in Other Countries A. There are wide variations in the uses of employment tests outside the U.S. 1. Asian examples of applications and frequencies a) Korea uses tests more than other Asian countries b) Japan hires directly from universities, so it does not rely on tests c) Japan does use FAST for hiring Japanese for U.S. expatriate work d) Few tests are used in Hong Kong, and Singapore e) China tests extensively 2. European examples of use and restrictions Switzerland includes graphology and astrology information In Italy, tests are heavily regulated 3. The U.S. and England are employment tests research and development centers XII. Bottom-line on Selection See Figure 6-1, p. 233 on the recommended chronology Define the job with a focus on job specifications (competencies) compatible with strategic goals and executing those goals Recruit from a broad pool of candidates Use valid initial screening devices Do background/ reference checks Use behavioral interviewing technique with structured format or independent multiple interviewers asking behavioral questions. Use weighting scheme for information Extend an offer XIII. Summary A. The importance of effective pre-employment testing is increasing B. Cognitive ability tests are valid but can result in adverse impact C. Personality tests have less adverse impact, but are typically less valid D. Drug tests used in a majority of firms may screen out less effective employees E. Negligent hiring lawsuits dictate that employers should use reference check F. Honesty tests are valid predictors G. Assessment centers and performance tests are accurate, more expensive but more defensible in court H. Most firms use multiple measures and a subjective weighting system for rank ordering I. Most firms use interviews at some point in the selection process J. The structured behavioral interview is quite valid K. The interview makes inferences about the applicant's ability to do the job L. Adoption of guidelines can minimize bias, boost validity and improve effectiveness of interview procedure M. Functional utility of the interview rests on the determination of 1. What factors are most accurately and consistently evaluated during interview 2. Can other selection procedures measure factors as well as or better N. Most effective selection place emphasis on interaction “matching” the person and organization and job O. Above all fairness and the avoidance of prejudice and illegal discrimination must be considered in the selection process CHAPTER 6 - IMPORTANT TERMS 16 Personality Factors (16PF) - Provides scores on the factors of the FFM; it is also used to screen applicants for counterproductive work behavior. Accomplishment Record - Candidates write about previous accomplishments. Achievement tests - Attempt to measure the effects of knowledge obtained in a standardized environment. Actuarial - Statistical decision making. Weights are applied to sources of information based on relative validity of the sources. Agreeableness/likeability - Five Factor Model personality factors that measures friendliness, cooperative vs. dominate. Assessment Center - A method that enables "standardized evaluation of behavior on multiple inputs. Multiple trained observers are used. Banding - Procedure of evaluating scores within a specific range to all be the same score (Like a letter grade of ‘A’ for scores between 90-100%) Behavioral interviewing - Applicants describe how they would behave in specific situations. Candidates are asked to describe actual experiences they have had in dealing with specific, job-related issues or challenges. This method yields superior validity. Bennett Mechanical Comprehension Test - Consists mainly of pictures depicting mechanical situations with questions pertaining to the situations. Biographical Information Blanks - BIBs are a lot like WABs, except that the items on a BIB tend to be more personal and experiential. Clinical or holistic prediction - calls for an overall judgment based on information gathered by experts. Cognitive ability tests - Measure one's aptitude or mental capacity to acquire knowledge based on the accumulation of learning from all possible sources. These tests are also known as general mental ability tests. Conscientiousness - Five Factor Model personality factors that measures dependability, carefulness. Content validity - Expert assessment of extent to which the content of a selection method is representative of the content of the job. Core Self-Evaluations (CSE) - Broad and general personality trait composed of four heavily researched traits: (1) self-esteem, the overall value that one places on oneself as an individual; (2) self-efficacy, an evaluation of how well one can perform across situations; (3) neuroticism, the tendency to focus on the negative; (4) locus of control, the extent to which one believes she or he has control over life’s events. Correlation coefficient - Summary index of the degree of relationship between the predictor and the criterion Cross-cultural Adaptability Inventory - Useful international selection test focusing on the ability to adapt to new situations and interact with people different from oneself Dimensions - Clusters of behaviors that are specific, observable, and verifiable and can be reliability and logically classified together. Discriminatory bias - In terms of interviews, discrimination based on stereotypes of gender, race, age, or disability Discriminatory impact - Interview questions or processes used which could result in adverse impact. Discriminatory intent - Non-job-related questions in interviews of only one group (protected) and not others. Emotional Intelligence (EI) - A multidimensional form or subset of social intelligence or a form of social literacy. EI is a set of abilities that enable individuals to recognize and understand their own emotions and those of others in order to guide their thinking and behavior to help them cope with the environment. Empirical or criterion-related validity - involves the statistical relationship between scores on some predictor or selection method and performance on some criterion measure. Employee Polygraph Protection Act- Federal law banned the use of the polygraph for selection and greatly restricts the use of the test for other employment situations. Extraversion/introversion - Five Factor Model personality factors that measures Outgoing/sociable Fair Credit Reporting Act (FCRA) - A federal law that regulates how background check agencies provide information about consumers. Five-factor Model of Personality (FFM) - Sometimes referred to as the "Big Five" measures: Introversion/extraversion, emotional stability, agreeableness/likeability, conscientiousness, and openness to experience. Foreign Assignment Selection Test (FAST) - assesses cultural flexibility, sociability, conflict resolution style, ad leadership style. Functional utility - In context of the interview, it is the practical assessment of the interview based on the factors that are the most accurate and the most consistent evaluated during the interview and whether other selection procedures measure those factors betters than the interview. Global Assignment Preparedness Survey - Useful and valid international selection test which assesses candidates on six dimensions Graphology- handwriting analysis Group/panel interviews - Multiple interviewers independently rate applicant during interview process. In-basket - Consists of a variety of materials of varying importance and priority that would typically be handled by an incumbent. It simulates a situation in which the candidate must deal with a number of memos and items accumulated in their in-basket. Incremental validity - The degree to which a measure improves the validity of decisions made from one source of information (e.g., behavioral interviews add incremental validity to predictions from GMA). Integrity tests - Measure attitudes and experiences related to a person’s honesty, dependability, and trustworthiness. Job Compatibility Questionnaire (JCQ) - Developed to determine whether an applicant's preferences for work characteristics match the characteristics of the job. Knowledge-based tests - Measure bodies of knowledge required by a job Leaderless Group Discussion - Small group problem assignments given to three to six people without appointment of a leader. Observer assesses dimensions such as oral communication, tolerance for stress, adaptability, leadership, and persuasiveness. Linear relationship - Higher scores on selection method (e.g., GMA) go with higher performance and lower scores go with lower performance through entire score continuum. Miner Sentence Completion Scale - Measures manager’s motivation to manage others. Minnesota Clerical Test - Requires test takers to quickly compare either name or numbers and to indicate pairs that are the same. Minnesota Multiphasic Personality Inventory - Used for jobs that concern the public safety or welfare; used to identify pathological problems in respondents, not to predict job effectiveness Negligent hiring - Holds an organization responsible for the behavior of employees when there is little or no attempt to assess critical characteristics of those who are hired or retained. Neuroticism - Five Factor Model personality factors that measures emotional stability Openness to experience - Imaginative, curious, experimenting Personality - Individual’s consistent pattern of behavior Performance tests - Measure KASOCs or competencies. Personnel Selection - The process of gathering and assessing information about job candidates in order to make decisions about personnel Physical environment - In this chapter, refers to the conditions surrounding the interview. Projective tests - Purpose and scoring procedure of test are disguised from the test taker. Examples include graphology, Rorschach inkblot test, Thematic Apperception Test (TAT), and Miner Sentence Completion Scale (MSCS). Race/Sex norming - Interpretation of test scores based on the test taker's race, ethnicity or sex. Rating bias - Applicant attributes, interviewer attributes, and situational attributes that can influence ratings. Reference checks - Provide information about an applicant’s past performance or measure the accuracy of applicants’ statements on their resumes. Reliability - Concerns the consistency of measurement. Role-Play Exercise - In the context of an assessment center, candidates assume role of incumbent and deal with a subordinate about a performance problem. Rorschach Inkblot test - Shows a series of inkblots to respondents who must then record what they see in each one. Safety Locus of Control Scale – a paper and pencil test containing 17 items assessing attitudes toward safety. Self-report Personality Inventories - Typically consist of a series of short statements concerning one's behaviors, thoughts, and emotions, past experiences, preferences, or characteristics. Examples include Gordon Personal-Inventory Profile, Guilford-Zimmerman Temperament Survey (GZTS), and Minnesota Multiphasic Personality Inventory (MMPI). Service Orientation Index (SOI) - Originally developed to predict the helpfulness of nurses aids. Also useful for clerical employees and truck drivers Situational interviews - Applicants given hypothetical situations and asked to describe how they would respond. Situational judgment tests - Measure a variety of skills with short scenarios asking test takers what would be their most likely response. Stereotyping - Categorizing groups according to general traits then attributing those traits to a member of the group. Structured interviews - Same questions to all candidates in the same order Tests of specific abilities - Measure such abilities as clerical, physical, or psychomotor. They assess factors such as eye-hand coordination, sensory skills, and mechanical ability. Thematic Apperception Test - A test that typically consists of 31 pictures that depict a variety of social and interpersonal situations Unstructured Interviews - Measure a variety of skills using questions that vary from candidate to candidate and interviewer to interviewer. Utility - Concerns the economic gains from using particular methods. Validity - Extent to which various selection methods successfully predict some important criterion such as performance on the job or employee turnover. Validity generalization- Extent to which test results can be applied to predicting job performance for other jobs Weighted Application Blanks - An application blank that can be scored like a paper-and-pencil tests Wechsler Adult Intelligence Scale - One of he most valid and heavily research personnel selection tests. Wonderlic Personnel Test - A 50 question personnel selection test; valid and more practical test. Work sample - Consists of tasks that represent the type, complexities, and difficulty level of activities that are actually required on the job. CHAPTER 6 IMPORTANT CASES Albemarle Paper Company v. Moody - clarified job-relatedness defense, requiring a careful job analysis to identify the specific knowledge, skills, and abilities necessary to perform the job. Also required employer to use specific criteria on which to evaluate job performance in showing job relatedness Connecticut v. Teal - all hurdles of employment selection that result in adverse impact must meet job relatedness test. Griggs v. Duke Power - established “disparate impact” theory Lewis v. City of Chicago - cutoff score of firefighter’s exam caused adverse impact and found not to be “job related.” CHAPTER 7 PERFORMANCE MANAGEMENT AND APPRAISAL Objectives After reading this chapter, you should be able to Understand the value and uses of performance appraisals in organizations. Know the prescriptions for more effective performance management and appraisal. Define performance and distinguish between performance and correlates of performance. Discuss the legal implications of performance appraisal. Explain the various errors and biases in ratings and proven methods to reduce them. Describe the ProMES system and report on its effects. Describe the necessary steps for implementing an effective appraisal feedback system. CHAPTER 7 –CHAPTER SUMMARY I. Overview. A. Performance appraisals are used by most organizations 1. High level of dissatisfaction “the appraisal of performance appraisal is not good” 2. Raters, ratees, and administrators all dissatisfied B. Performance appraisal systems are a component of high-performance work practices C. Performance appraisals are heavily litigated 1. Legal grounds for challenging appraisal systems are expanding D. Prescriptions for Effective Performance Management 1. Strive for precision in defining and measuring performance dimensions 2. Derive content from internal and external customer requirements in the context of organizational objectives 3. Derive ratings from multiple raters 4. Correct the effects of situational constraints E. Performance Management and appraisal is a continuous process II. How Do We Define Performance and Why Do We Measure It? A. Performance is the record of outcomes produced on a specified job function or activities during a specified time period 1. Performance as a whole is equal to the sum of performances on job functions 2. Performance Appraisal (PA) should not focus on the traits or personal characteristics of the performers 3. Distinguish between the performers knowledge, skills and competencies and actual performance outcomes 4. Skills, knowledge and competencies indicate potential to perform 5. Performance should be measured within the context of organizational objectives B. What Are the Uses for Performance Data? 1. Performance Management and Compensation 2. Internal Staffing a) Promotions and terminations 3. Training Needs Analysis 4. Research and Evaluation a) Validation research III. Legal Issues Associated with Performance Appraisals A. Appraisal data is subject to considerable legal scrutiny Charges of unfairness and bias can result from decisions made using PA data Seek relief through federal law including CRA and ADEA B. Suggestions recommended for legal defensibility (see Fig. 7-3, p. 246) 1. Fig 7-3 lists 15 PA characteristics related to the content, process and results of PA 2. Characteristics presented in approximate order of importance in prediction of court case outcomes 3. Violation of the 80% rule most important indicator of negative employer outcome 4. Focus on “excessive subjectivity” 5. Conduct audits of PA systems data IV. Designing an Appraisal System A. Measurement Content 1. The focus of the appraisal a) Combination of person-oriented, focus is on person’s characteristics & work-oriented, focusing on the record of outcomes b) Effective performance appraisals focus on the record of outcomes 2. 6 Categories of Performance Outcomes a) Quality b) Quantity c) Timeliness d) Cost effectiveness e) Need for supervision f) Interpersonal impact/ contextual or citizenship performance 3. Overall PA is less accurate rating than those done on each relevant criterion for each job activity and provides less specific feedback to the performer a) Greater specificity/precision in defining criteria, the more effective the appraisal B. The Measurement Process 1. Three basic ways raters can make performance assessments a) Comparisons among ratees’ performance b) Comparisons among performance level anchors or standards c) Comparisons of individuals’ performance to anchors or standards 2. Comparisons among Ratees' Performances a) “Paired comparisons” compares all possible pairs of ratees on “overall performance” or some other standard b) “Straight Ranking”: or “Rank Ordering” creates a list of employees from best to worst based on performance c) “Forced Distribution” requires that a designated portion of ratees fall into the identified categories of ratings, similar to a bell shaped curve i) Once popular method (Ford, GE) ii) Recent research is not favorable (dropped by Microsoft in 2008) 3. Comparisons among Performance-Level Anchors a) Computerized adaptive rating scales (CARS): presents raters with pairs of behavioral statements reflecting different levels of performance on the same performance dimension b) Forced choice: requires the rater to compare performance statements and select one (or more) as the most descriptive i) Hidden scoring key ii) Some evidence deliberate bias can be reduced iii) Raters do not like this method because of the hidden scoring key 4. Comparison to Performance-Level Anchors a) Graphic rating scales: use adjectives or numbers as anchors but the descriptive detail of the anchors differs widely b) Behaviorally anchored rating scales (BARS): graphic scales with specific behavioral descriptions defining various points along the scale for each dimension i) Recording observations on scale can improve performance compared to other methods c) Summated scales i) Behavioral Observation Scales (BOS) the rater is asked to indicate how frequently the ratee has performed each of the listed behaviors. The ratings are then averaged or totaled for each person rated. d) Performance distribution assessment (PDA) i) Only method that statistically incorporates constraints on performance as a formal part of the measurement process e) Management by objectives (MBO), comparison between specific, quantifiable target goals and actual results achieved by an employee i) Effective when precise definitions of objectives are used (Figure 7-4) ii) Mutually agreed upon by employee and supervisor iii) Popular for managerial PA What Is the ‘Bottom Line” on What It Is We Should be Measuring Focus on the record of outcomes vital to strategic goals of organization Recommendation: Rate outcomes on relative frequency Customer defines levels of performance, then raters indicate the percentage of times the rate achieved a level of performance relative to all the opportunities s/he had to achieve that level of performance Relative frequency superior to ratings of intensity or satisfactoriness Ratings of relative frequency result in higher levels of reliability and the people who receive feedback find them more helpful c) Tie performance measures to job description C. Control of Rating Errors and Bias 1. Leniency/Severity- leniency is an intentional error, ratings for employees are at the high end of the rating scale regardless of the actual performance a) Most serious problem with appraisals whenever they are linked to important decisions such as compensation b) Primary reason why companies have turned to forced distribution. c) Leniency related to the characteristics of the rater-Low Conscientious and High Agreeableness d) Self-efficacy training for raters reduces leniency 2. Halo/Horns effect – not deliberate; rater allows a rating on one dimension (or an overall impression) for an employee to influence the ratings assigned to other dimensions 3. Central Tendency – deliberate error; ratings for employees tend to be toward the center (midpoint) of the scale regardless of the actual performance 4. Fundamental attribution error/ Actor-observer Bias a) Actor-observer bias- tendency of raters to underestimate external factors and for ratees to overestimate those same factors b) Major factor causing perception of unfairness, not appropriately considering situational constraints (Figure 7-9) c) Training can reduce the error/bias d) Goal based PA with performance dimension/constraint matrix 5. Representativeness- judgments about people (or their performance) based on their similarity to people who exhibited prominent or memorable levels on the attribute being judged, even though the similarity may have no causal connection to the attribute; difficult to overcome 6. Availability - mistaking the ease with which a category of outcomes can be recalled as an indication of its frequency of occurrence relative to other categories 7. Anchoring - insufficiently altering a judgment away from some starting point when new information is received a) Training can help reduce this error D. Rater Training 1. Unintentional errors a) Frame of Reference training improves rater accuracy, controls unintentional errors 2. Intentional errors a) Administrative changes (e.g. forced distribution, hidden scoring keys) attempt to control intentional bias b) Best to hold raters accountable for their ratings and use more than one “qualified” rater E. Defining the Rater 1. Ratees 2. Supervisors – most companies give full responsibility of appraisals 3. Peers 4. Clients, customers, mystery shoppers 5. Higher-level managers 6. Using multiple raters (360- degree) is beneficial for greater accuracy, fewer biases, perceptions of fairness, and fewer legal suits a) 360 degreee appraisal is a “High Performance Work Practice” F. Defining the “Ratee” 1. Potentially includes individual, work group, division or organization wide performance 2. Two Conditions for measuring performance at the aggregated level a) High Work Group Cohesiveness b) Interdependency, when it is difficult to identifying individual contributions 3. Process tracing software provides data on the interactions and contributions of individual team members 4. Match the level of appraisal with the objective being appraised G. The Productivity Measurement and Enhancement System (ProMES) 1. Method to measure aggregated performance 2. Goal to improve unit productivity or overall performance 3. Designed to give employees precise performance feedback developing sense of ownership and empowerment 4. Research indicates this method improves performance, job satisfaction, and reduces job stress 5. Steps of ProMES similar to MBO, outlined in Figure 7-13 p 265 6. Critical to identify relative importance of performance indicators, this can help identify priorities for improvement H. Benchmarking- Gauging internal practices to external standards 1. Process of measuring internal practices against an external reference/ standard 2. Most often used at the aggregate level 3. Critical to have top management support and commitment 4. When radical change is necessary, use the “shaping” process to facilitate gradual, sustained improvement Identify what is to be benchmarked Identify comparable companies Collect data to define goal Collect data to determine current performance level Reduce target to measurable steps Train to meet sub-goals Provide feedback and use rewards Increase sub-goals to get closer to target goals Recalibrate benchmarks I. Administrative Characteristics 1. Issues such as frequency and timing are outlined in Figure 7-16 p 269 2. Many PA’s are computer adaptive or completed are on-line Performance monitoring, is it an invasion of privacy or a way to gather information about performance, particularly when work is performed at remote locations J. Methods of Delivering Performance Feedback 1. Provide specific and timely feedback 2. Supportive feedback provides motivation 3. Greater precision in feedback more likely leads to improved performance 4. Characteristics of effective feedback a) Working tasks are familiar to the ratee b) Performance cues support performance c) Includes information on discrepancies between performance and precisely defined performance standards d) Not threatening to the ratee psychologically 5. Recommendations for feedback sessions a) Avoid being disturbed b) Make sufficient time c) Keep notes on effective and ineffective behavior d) Be relaxed and informal e) Include praise f) Include employee’s self assessment g) Include supervisor’s response to employee’s assessment h) Include action plans i) Consider situational constraints j) Discuss employee’s career aspirations and developmental goals k) Be clear, specific, descriptive and job related l) Occur frequently and timely 6. Strength based PA combined with goal setting focuses on employee strengths and concentrates on prevention focused behaviors V. Summary A. Performance appraisals are important tools 1. Managing and improving the performance of employees 2. Making valid staffing decisions 3. Enhancing the overall effectiveness of the firm's products and services B. Performance appraisal systems must be: 1. Carefully developed 2. Monitored 3. Revised as needed 4. Linked with other HR functions, such as determining training CHAPTER 7 - IMPORTANT TERMS 360 degree appraisal (360) - The use of multiple, qualified raters; another HPWP. Actor/observer bias - raters tend to underestimate the role of situational factors and actors/ratees to overestimate their role in less than perfect performance Anchoring - insufficiently altering an initial judgment away from some starting point when new contradictory information is received Availability bias – concluding that the ease with which outcomes can be recalled is an indication of their frequency of occurrence relative to other categories; the tendency to believe that the outcomes you remember are also the outcomes that were most frequent Behaviorally anchored rating scales (BARS) - a graphic rating scale with specific behavioral description defining various points along the scale for each dimension Behavioral observation scales (BOS) – rater is asked to indicate how frequently the ratee performed a list of behaviors Benchmarking - process of gauging internal practices to external reference or standard; measuring a company’s performance as compared against the best Central tendency - ratings tend to be at the center of the scale regardless of employee performance Comparisons among performance level anchors - selects the anchor that is most descriptive of the person being appraised Comparisons among ratee's performances - comparing ratees to each other through ranking, paired comparisons, or forced distribution. Comparisons to performance level anchors - compares individuals to predetermined definitions of performance Computerized Adaptive Rating Scales (CARS) - rating method presents raters with pairs of behavioral statements related to performance and the rater picks the most descriptive one. Pairs of statements are presented until a reliable rating can be determined. Contextual performance contributions – factors facilitating a pleasant work environment, and compliance with organizational and subunit policies Frame of Reference Training - designed to reduce unintentional rating error by familiarizing raters with what behaviors or outcomes to consider when making their ratings, allowing for practice making ratings, and providing feedback on their practice Forced choice - the rater makes comparisons among anchors or performance level anchors for a job activity Forced distribution - presents the rater with a limited number of categories and requires the rater to place a designated portion of the ratees into each category similar to a bell shaped curve Fundamental attribution error - attributing observed behaviors or outcomes to the disposition of the person being observed while underestimating the causal role of factors beyond the control of the performer Graphic ratings scales - requires the rater to make comparisons of the employee’s performance to specified anchors or standards of performance Halo/horns effect - allowing the rating of one dimension for an employee to influence ratings of other unrelated dimensions Intentional bias - errors committed because rater is in some way motivated to provide an inaccurate rating, for example giving a higher rating to avoid a negative discussion with an employee Inter-rater reliability - extent to which independent raters agree on an evaluation of performance Leniency/severity error – occurs when ratings are generally at the high end of the scale regardless of actual performance Management by objectives (MBO) - calls for comparison between specific, quantifiable target goals and the actual results achieved by an employee Mixed standard scales – similar to the forced-choice approach, attempts to control deliberate response bias by raters. MSS usually consist of sets of three statements that describe high, medium, and low levels of performance on a job activity or dimension. Paired comparisons - comparing all possible pairs of ratees on "overall performance" or some other standard Performance - record of outcomes produced on a specified job function or activity during a specified time period Performance Distribution Assessment - rating method that statistically incorporates constraints on performance as a formal part of the measurement process Performance level anchors - brief descriptions of levels of performance (sometimes referred to as "descriptors") Performance monitoring - automated review of an employee’s performance and behaviors, for instance with cameras Person-oriented appraisal - focusing on the traits of the person who performed the behavior Primacy or recency effect - ratings are heavily influenced by ratee's behavior during early or late stages of the review period ProMES – a measurement of performance similar to MBO focused on aggregated, unit performance Ratee – the individual, work group, division or organization being appraised Rater – the individual providing the evaluation of performance Relative frequencies - defines level of performance and rates how frequently the ratee met that level of performance in the context of all times that they had an opportunity to do so Representativeness error - making judgments about people (or their performance) on the basis of their similarity to people who exhibited prominent or memorable levels on the attributes being judged Shaping – a behavioral change technique that promotes gradual improvement from a known level to a desired level Situational constraints – factors presumably outside of the control of the individual being rated (e.g. the quality of raw materials) Straight Ranking – the manager list employees from best to worst Summated scales – includes the behavioral observation scale (BOS), the rater is asked to indicate how frequently the ratee has performed each of the listed behaviors Unintentional errors - errors committed because rater lacks the necessary skills to make accurate ratings Work-oriented appraisal - focusing on the record of outcomes that a person achieved on the job; most effective approach v. person-oriented appraisal Instructor Manual for Human Resource Management John H. Bernardin, Joyce E. A. Russell 9780078029165, 9780071326186
Download
Close
Close
