CH-8 Evaluating and Closing the Project – Project Management in Practice : Book Guide

Chapter 8 Evaluating and Closing the Project This chapter discusses the issue of appraising the progress of a project and its performance and describes the various ways that projects can be closed. Project evaluation criteria are listed and discussed. Also, the formal process of auditing a project is overviewed including the information that should be contained in the audit report. Guidelines for when to consider closing a project, the types of project closure, and the closure process itself are also discussed. The chapter concludes with a discussion of the project final report. Cases and Readings Some cases appropriate to the subject of this chapter are: Harvard: 9-696-083 Biogen, Inc.: rBeta Interferon Manufacturing Process Development This 18-page case concerns a biotech firm that is wrapping up a new manufacturing process project for a newly developed drug. One of the senior scientists is charged with performing an evaluation of the project to document the lessons learned and provide guidance for future projects. Harvard: 9-692-069 Motorola, Inc.: Bandit Pager Project This 17-page case places the student in the role of conducting an evaluation of an automated production process for radio pagers. Although considered a major success by the company, there appear to be some difficulties in marketing and production that need more attention. A 19 page teaching note (5-692-069) is available. Harvard: 9-175-184 Northeast Research Laboratory (B) This excellent 17-page retrospective case describes a client’s unexpected cancellation of what was thought to be a successfully on-going project. Involves issues of control, auditing, evaluation, termination, behavior, leadership, and organization. Some readings appropriate to the subject of this chapter are: B.M. Staw et al. Knowing When to Pull the Plug (Harvard Business Review, March-April 1987). This article has already become a classic on the psychological basis for continuing a project invariably headed for disaster. The authors clearly point out the dangers of certain personality types in project managers matching the needs profile of certain organizations. In these all-too-frequent situations, projects are continued way past the point where common sense would have called for their termination. Royer, I., Why Bad Projects are So Hard to Kill, (Harvard Business Review, February, 2003, 49-56). This reading asks the fundamental question of why managers keep pushing forward on a project even when faced with mounting evidence that success is pretty well unachievable. The article suggests that it’s not managerial incompetence or entrenched bureaucracy that makes projects hard to kill, but “a fervent and widespread belief among managers in the inevitability of their projects’ ultimate success.” This belief of success can spread from the project champion all the way up to top management and can lead to irrational behavior when it comes to project decisions. The reading offers suggestions to avoid the “blind faith” that is sometimes associated with failing projects. Answers to Review Questions 1. What are some of the commonly used project evaluation criteria? Answer: Commonly used project evaluation criteria include: • Original criteria for selecting and funding the project • Success to date (schedule and budget efficiency, customer impact/satisfaction, and business success) 2. What are the measurement challenges associated with using a multivariate model such as a scoring model? Answer: Multivariate models often include some subjective criteria which may be more difficult to measure. 3. Contrast project audits with financial audits. Answer: 4. What are the typical steps in a project audit? Answer: The typical steps in a project audit are: • Familiarize the audit team with the requirements of the project, including its basis for selection and any special changes by upper management. • Audit the project on-site. • Write up the audit report in the required format. • Distribute the report. 5. What information should be included in a project audit? Answer: At a minimum the audit report should contain the following: • Introduction • Current status • Future project status • Critical management issues • Risk analysis and risk management • Final comments 6. What are the types of project closure? Answer: The types of project closure include: • Project extinction • Closure by addition • Closure by integration • Closure by starvation 7. What are the main duties of the closure manager? Answer: 8. What issues should be addressed in the project final report? Answer: The following issues should be addressed in the project final report: • Project performance • Administrative performance • Organizational structure • Project teamwork • Project management techniques 9. What are the major purposes served by an after-the-fact project evaluation? Answer: The major value of a post-project audit is to discuss (and inform senior management): • Poor practices that should be avoided in future projects, and • Beneficial practices that should be adopted by future projects. An after-the-fact evaluation may be required by a customer who wishes to manage future relationships with the firm. Suggested Answers to Discussion Questions 1. How should an audit team handle an audit where it is explicitly restricted from accessing certain materials and/or personnel? Answer: For an effective audit, the team must have access to all project records and personnel, as well as all functional and other organizational area records and personnel. Only access to the customer should be limited to require prior clearance from top management. If access is denied by team members, the audit team may have to appeal to the project manager. If the project manager is the cause for the problem, the audit team may have to appeal to senior management. In cases where full access is not provided, the results of the audit must be qualified noting the lack of access. 2. What might be some characteristics of a good closure manager? Answer: Among some of the desirable characteristics are: • Having familiarity with the administrative requirements of project terminations. • Being objective and impartial. • Being calm under stress. • Being personable. • Being task rather than functionally oriented, and • Having the ability to effectively and efficiently execute the eight termination tasks. 3. How should a committee determine which closure method to use? Answer: As indicated in the text, there are four types of project closures: • Project extinction … when the project is either successfully completed or the expectation of failure is high. • Termination-by-addition … when it is an “in-house” project which is successfully completed and institutionalized as a formal part of the organization. • Termination-by-integration … when the output of the project becomes a standard part of the operating systems of the sponsoring firm or client. • Termination-by-starvation … when it is politically unacceptable to terminate the project so it is left to wither until it is a project in name only. The termination method selected will depend primarily on what is to happen with the project. For example, if it is to be an ongoing entity, addition or integration is best. If its job is complete, then extinction is best. Termination-by-starvation is not an ideal method since it leaves the project on the books. 4. What steps might be taken to reduce the anxiety of project team members facing an audit? Answer: The focus of the audit must be positive with the reports kept constructive, unemotional, and professional. Pre-release discussions and a formal "appeal" procedure for disagreements are also helpful. 5. What are the dangers of evaluating a project based on the reason(s) it was selected (such as being a competitive necessity), rather than the goals and objectives in the project proposal or contract? Answer: The reasons for project selection are often more general in nature while the project’s goals are much more specific. One danger of evaluating the project based on the reason it was selected is that a change or shift in the project’s specific goals could go undetected. This could result in a situation where project’s goals become out of sync with broader organizational goals. It might also lead to project proposals written in such a way as to minimize project goals or expectations, yet high enough to insure selection. In spite of these dangers, the criterion for project selection is a good, but not sufficient, tool for evaluating completed projects as well as for helping to determine if a project should be terminated before completion. 6. It is frequently suggested that items that will become potential problems later in the project life be decided up front, such as how to allocate revenues or dispose of project assets. How can this wisdom be used for project problems that cannot be foreseen? Answer: Major risks associated with the project and their potential impact on the project’s schedule, cost, or performance should be identified. If alternative courses of action exist that may change future risks, they should also be identified. The parent organization should also insist on proper ethical standards of behavior from the project manager, team, and all functional managers connected with the project. A large majority of the problems referred to in this question concern credit-for-success and blame-for-failure. If a project succeeds or fails, the success or failure belongs to the entire team. 7. Can you think of any acceptable ways of assigning credit for profits (or responsibility for costs) resulting from a project on which several departments worked? Answer: There is no theoretically correct way of dealing with these problems. Any method of sharing costs and/or profits that is acceptable to the several functional managers is acceptable. The method of cost/profit sharing should be accepted prior to initiating the project. Attempts to solve the problem when the project is completed are doomed. 8. How might an audit team deal with an attempt to co-opt the team? Answer: “Co-opt” means “to appropriate as one’s own.” This is a common tactic for parties being audited. Passive resistance is advised. The audit team should listen with interest to all complaints, advice, etc., and proceed as if the co-opting had not occurred. A noncommittal attitude is best for auditors. 9. In Section 8.2, subsection The Audit Report, six elements are listed that should be included in every audit report. For each element, explain why its inclusion is important. Answer: 10. Referring to the “Evaluation Criteria” subsection of Section 8.1, how do you reconcile the difference between the goals and objectives of the project as given to the project manager in Chapter 1 and these criteria by which she or he is now judged? Answer: The initial goals and objectives set for a project are determined prior to the start of the project. While they are, as indicated, often changed during the project’s life (and relative to other similar projects) they are generally associated with the initial criteria. However, with the passage of time and the completion of the project, other factors come into play and, as a result, the PM is often evaluated against a broader set of criteria including such factors as the project’s apparent “success” in terms of the project’s efficiency in meeting the budget and schedule, the level of customer impact/satisfaction, its business/direct success, and future potential. None of these four criteria could have been established prior to the start of the project. There are, in fact, post hoc criteria. 11. If the actual termination of a project is a project in itself, how is it different from other projects? Answer: The termination project is similar to any other project except that: • There is only one person conducting the termination itself and there is no "project selection" phase. • It is a process of distribution of resources and personnel rather than acquisition of them. • The tasks to be conducted are relatively standard or else specified in advance. • It is best conducted by someone without an emotional investment in the project, and • It is a completely administrative project with a single purpose. 12. What are some reasons that a failing project might still not be terminated? Answer: There are often political, psychological, and other reasons why projects are continued when they should be terminated. Sometimes, though rarely, projects that fail to meet their objectives are continued because they are accomplishing other things of value to the organization. 13. What might make a project unsuccessful during the termination process? Answer: Any aspect of termination may cause an otherwise successful project to become a failure such as: • An inability to complete the work. • Not delivering the desired result. • Providing poor product support, etc. 14. Who would make the best auditors: outside unbiased auditors or inside auditors who are more familiar with the organization, its procedures, and the project? Why? Answer: There are pros and cons to both options. Inside auditors would probably be more efficient and less expensive. However, they would not be as objective and they may not be familiar with how other firms or organizations operate and could not make accurate comparisons. External auditors would be more expensive. However, they would probably be able to better compare the outcome of the project with that of other firms in the industry. For example, internal auditors may positively evaluate a situation that appears favorable from the company’s perspective when outside auditors would report that the situation is not up to market standards based on industry experience. 15. Under what circumstances would you change your answer to the previous question? Answer: Students who initially selected external auditors based on perhaps higher objectivity might note that in situations when cost or time is critical, internal auditors might be preferred. Students who favored internal auditors might recognize the need for external auditors when objectivity or external market expertise is the key concern. If there's a significant change in the organization's needs, such as a shift in the level of complexity of the project, the emergence of new industry standards, or a change in the organization's resources that affects the cost or effectiveness of internal versus external auditors. 16. Would frequent brief evaluations be best, or would less frequent major evaluations be preferred? Answer: Frequent, brief evaluations have a better chance of finding problems as they are developing and can thus be solved easier and quickly. The frequent, low level evaluations are also less disruptive to the project team. Because of their frequency, they would also better lend themselves to use as a control device. 17. Should the results of an evaluation or audit be shared with the project team? Answer: Except for any confidential personnel items, the project team members should have access to the full report. Without feedback, the team members are limited in their growth possibilities and could be demotivated to work on future projects. 18. Under what circumstances is a detailed audit apt to be useful? Answer: When a general audit exposes a serious problem in the project or in the methods by which the project was conducted, a detailed audit may be required to study such problems in greater depth and recommend solutions. Incidents for Discussion Suggested Answers General Construction Company Question: If you were the project manager, how would you handle the situation? Answer: The member of the auditing team is not following appropriate procedure in contacting the customer directly. The PM should immediately inform top management of both the technical problem with the contract and of the breach of audit procedure. Then the PM should: • Contact the client to resolve the problem. • Discipline the auditor for his breach of procedure. How can a customer be assured of satisfactory project completion? Answer: The customer should: • Include such an audit process in the contract, with results coming directly to them, and • Require that the audit be conducted by their own people, or an external entity. Lexi Electronics Question: which of the three methods would you recommend, and why? Answer: The case specifies three methods: extinction, addition, and integration. Since the project has been successfully completed, addition is the method of choice. The project team members should be utilized to lead, at least initially, the production of the new phone. They have the most familiarity with the process and the product. The Shocking Machine Question: what do you suggest Mr. Lament do, if anything, to begin his preparation for defending or settling this potential lawsuit? Answer: As the corporate attorney for Lithotropic, Mr. Lament needs to: • Conduct a thorough evaluation of the project. • Evaluate the project’s performance relative to the initial plan and project objective. • Review the minutes of any project meetings with the client, this should cover the entire history of the project from the original launch meeting until the end of the project. • Review any change orders to the specifications or steps that are documented. • Review the monitor and control systems that were in place for the project. • Examine any documented communications between the project team and the client. • Meet with the project manager to see if a project diary was kept on this project. Once he has fully familiarized himself with the history of the project through this evaluation process he will know better how to proceed if the hospital actually sues the Lithotropic Corporation. Suggested Case Analyses and Solutions Friendly Assisted Living Facility -- 8 Teaching Purpose: This installment concludes the St. Dismas case. Issues include estimating the final construction budget, addressing a work slowdown, revising the Gantt chart, and the reporting of staff expenses. Question 1: Estimate the final construction budget. Answer: Budgeted Estimated Actual at Completion Remaining $ Construction Costs: Building 6,743,000.00 $6,743,000.00 $0.00 Contingency 674,300.00 $674,300.00 $0.00 Land 600,000.00 $600,000.00 $0.00 Program development & equipment costs 405,000.00 $405,000.00 $0.00 Furniture 400,000.00 $400,000.00 $0.00 A&E Fees @ 5% 347,000.00 $347,000.00 $0.00 Financing costs 202,000.00 $202,000.00 $0.00 Capitalized interest 135,000.00 $135,000.00 $0.00 Site improvements 125,000.00 $147,655.00 -$22,655.00 Phone & IS system 30,000.00 $30,000.00 $0.00 Kitchen equipment 30,000.00 $23,776.00 $6,224.00 Total 9,691,300.00 $9,707,731.00 -$16,431.00 Organizational Costs: Legal and accounting 25,000.00 $0.00 $25,000.00 Initial marketing 250,000.00 $0.00 $250,000.00 Project consultant 80,000.00 $0.00 $80,000.00 Follow-up Market survey 20,000.00 $0.00 $20,000.00 Total 375,000.00 $0.00 $375,000.00 Total 10,066,300.00 $9,707,731.00 $358,569.00 The organizational costs cannot be reflected in the final budget until the CFO explains when and how the expenses will be allocated back to the project’s budget for the expenses incurred by the parent organization. The above reflects the information provided in the case. Question 2: What should they do about the work slowdown? Answer: The work slowdown is probably being caused by the construction crew because they have not been informed of their next assignment. They are concerned about the lack of work after the facility is completed. The project manager needs to urge the contractor to inform his crew of their next assignment. Kyle also needs to insist that no additional slowdowns be allowed, and that tighter controls are put on the system to monitor that the project stays on schedule during the last few months. Question 3: Create a Gantt chart with the revisions to the duration of the interiors step. When will the project be completed? Answer: According to this revised Gantt chart, the project will be completed 8/1/01. Question 4: What is the importance of the fact that the Hospital staff expenses were not reported under the project budget? Answer: The importance of the omission of expenses on the project is significant. The CFO should have provided the PM with the overhead allocations at the beginning of the project. Those expenses can and will affect the project’s budget and need to be included in the overall project. The PM may not have control over the expenses but they affect his bottom line and the success of the project in meeting its budget. This late in the project there is little anyone, include the CEO, can do to have an impact on the success of the project budget without knowing these costs. Question 5: Write a one-page audit report of the project. Answer: The following is a sample audit report that follows the format of Figure 8.2. Datatech’s Audit Teaching Purpose: This case illustrates how many of the topics discussed in previous chapters can contribute to project failure and perhaps the need to terminate the project early. Question 1: To what extent were the problems facing the notebook computer development project avoidable? Answer: All of the problems noted in the case were avoidable. What could have been done to avoid these problems? Steve could have taken the following steps to avoid these problems. He could have: • Begun by doing some project planning. His failure to do so reflected the fact that he didn’t recognize the significant differences between a minor product line extension project and a more substantial new product line development project. • Involved the team in developing the WBS. Had the team been involved, the need for a hinge system and other unanticipated activities would have likely been identified. • Involved the manufacturing group earlier in the project. This would almost certainly have identified their inability to produce the notebook computer design. • Introduced a formal change request process. This would have helped with the VP’s request to redesign the computer to work with the new tablet computer. • Not agreeing to the project deadline without doing some initial planning. Question 2: Would it make sense to apply a project selection model such as the weighted scoring model to the project to determine if it should be terminated? Answer: Absolutely. Project selection models can be used to determine if a project should be continued. When used in this fashion, updated information can be entered into the model and a decision can be made regarding whether it makes sense to continue this project, particularly in light of new opportunities the organization may have. Question 3: In your opinion, are the types of problems that arose in this situation typical of other organizations? If so, what can organizations in general do to avoid these types of problems? Answer: These problems are indeed common. In some organizations, project managers that spend little time planning a project are referred to as “cowboys.” Besides the problem of inadequate planning, the problems of little team involvement, little cross-functional integration, and lack of processes to control change are also all too common. As discussed above, the solution of these problems is to ensure adequate project planning is carried out, that there are adequate levels of involvement both within the team and across functions, and to manage the scope of the project. Ivory Tower Systems Teaching Purpose: The big issues are should the project be continued, if so, how to manage the project going forward, and, if not, how to shut down the project. Question 1: Do you have any concerns about the audit process at ITS? If so, what are they and what changes to their auditing process would you recommend for the future? Answer: There are several concerns: First, there does not appear to be a formal audit process. In the present case, the audit was not initiated until two months prior to the projects deadline. The usefulness of audits is severely diminished if they are not conducted at points in the project where there is time to take corrective measures. Second, the auditor failed to follow up when Mr. Gilbert failed to provide requested documentation. Mr. Gilbert may have refused to provide them because they did not exist, as the auditor assumed, or he may be hiding more serious problems or even embezzlement. Without perusing the requested documents, the auditor cannot tell for sure. Third, the comment by one team member that Mr. Gilbert spends all his time on technical issues and little or none to managing the project was not followed up with other team members or by a conservation with Mr. Gilbert. This is superficial at best. Finally, the recommendation for senior management to consider terminating the project is a good one but without more data on the status of the project and the cost to continue, senior management will be unable to make an informed decision. The entire audit seems to be superficial at best. The auditor either needs additional training or needs to be replaced by someone able to dig deeper. Question 2: Critique the audit report. What are its strengths and weaknesses? Answer: • Perhaps the most serious criticism of the audit report is its tone. As is mentioned in the chapter, audit reports should be written in a constructive tone. Likewise, negative comments about individuals are to be avoided and comments made in confidence should not be included. • The current status section mentions that several significant technical challenges remain to be resolved but none are enumerated. Likewise, it hints of future problems that are also not enumerated. Ideally the audit would facilitate a comparison between planned and actual results. • The future project status section states there is virtually no chance the project will be on time but does not provide any estimate of a new completion date. • As mentioned above, the critical management issues section points out a management problem with Mr. Gilbert but does not dig deep into the issue. Additionally, no effort was made to obtain the documents that Mr. Gilbert refused to provide or to ascertain why he would not provide them. • As mentioned above, the final comments section recommends that senior management consider terminating the project without giving the information they would need to make that decision. Question 3: What are the pros and cons of using an external auditor in this situation? Answer: Pros: • An external auditor can dig deep and press for answers without affecting the relationships between coworkers; although, it does not appear that this auditor tried to dig for answers. • An external auditor can bring expertise to the audit that the organization does not have; although, this auditor does not demonstrate the required level of expertise. • An external auditor can do the work without interfering with the other projects. That is, the organization does not have to pull someone off another project to perform the audit. Cons • Workers may not confide in an external auditor the way they would someone they knew and trusted. The fact that only one worker complained about Mr. Gilbert would seem to indicate this. • An external auditor may not know the personalities and politics involved. In this case, the external auditor does not know for sure that the one worker who complained about Mr. Gilbert is reliable. • An external auditor may not know the types of information that senior management might want before making a tough call. Which type of auditor would you recommend for this project? In this case, the project is in serious trouble. Given that, an external auditor might be best; although, one with more skills and experience than the one used. However, since a big issue is the performance of Mr. Gilbert, a case can be made for using an internal auditor who mike know his strengths and weaknesses and know alternative ways of obtaining the needed documents. Solution Manual for Project Management in Practice Jack R. Meredith , Scott M. Shafer , Samuel J. Mantel 9781119385622