CHAPTER 11 TEST BANK
Evaluation Research and Problem Analysis
TRUE/FALSE
1. Like survey designs and field studies, evaluation research refers to a research method, not a
specific research purpose.
Answer: False
2. Policy analysis focuses upon whether the intended result was produced.
Answer: False
3. The policy process, like the research process in general, is fluid and does not always “start
at the beginning and conclude at the end.”
Answer: True
4. Outputs refer to the basic question of what a policy action is trying to achieve.
Answer: False
5. Goal-oriented public policies cannot be viewed as “if-then” statements: if some policy
action is taken, then we expect some result to be produced.
Answer: False
6. Applied studies such as program evaluation is that they do not require the same degree of
planning as do survey designs.
Answer: False
7. Evaluation apprehension occurs when officials fear that their own job performance is not
being rated.
Answer: False
8. In evaluation research, outcome measures serve as the independent variables.
Answer: False
9. Not only is it necessary to measure dependent variables in impact assessment, the
researcher also needs to measure the context within which the program is conducted.
Answer: True
10. Quasi-experiments differ from true experiments in that they lack random assignment to
control and experimental groups.
Answer: True
MULTIPLE CHOICE
1. Which statement is false with regard to the use of evaluation research?
a. it is gaining in popularity among researchers that really want to make a difference
b. federal requirements for program evaluations often accompany the implementation of new
programs
c. funding is available for program evaluations
d. Evaluation research may only use experimental design.
Answer: D
2. Which action does not represent a step in the policy analysis process?
a. there is a demand for some new course of action or strong opposition to an existing policy
b. formulate a hypothesis that will allow for exploration
c. identify the ultimate goals and the different actions for achieving those goals
d. consider the impact of the policy outputs
Answer: B
3. Which of the following is not a variable-oriented strategy for research?
a. realist approach
b. experimental approach
c. case-study approach
d. all of these
Answer: B
4. Program evaluation is designed to link the intended actions and goals of criminal justice
policy to ____.
a. empirical evidence that supports them having the desired effects
b. policies that are made to benefit communities
c. implementation of policies
d. randomized field experiments
Answer: A
5. Problem analysis takes place in the earlier stages of the policy process. In contrast,
program evaluation studies are conducted ____.
a. in the interim stages
b. In the later stages
c. In the preliminary stages
d. In the middle
Answer: B
6. Which statement is inaccurate?
a. impact assessments evaluate whether or not specified goals are attained
b. process evaluations focus on whether programs are carried out according to plans
c. problem analysis usually considers a limited range of alternative choices
d. ideally, impact assessments and process evaluations are conducted separately
Answer: D
7. Which of the following does not represent potential problems in doing evaluation research?
a. conflicting goals
b. vague goals
c. when the intended actions and goals are linked through empirical evidence.
d. difficulty in expressing policy goals in the “if-then” form that can be empirically tested
Answer: C
8. Which step is of least concern in the evaluability assessment?
a. determine the degree of support for your project
b. determine what the general program goals are as well as the specific objectives
c. determine how the goals are translated into the program components
d. determine whether people in the organization have some sort of direct or indirect stake in
the program
Answer: A
9. One of the most important parts of conducting evaluation research is _____.
a. selecting a sample that will support the program goals
b. formulating goal statements that are empirically testable
c. gathering official documents since that is where the researcher can find complete
descriptions of goals
d. making sure that the staff of the agency where you conduct research like you by doing
anything to get them on your side
Answer: B
10. All of the following would be examples of goal statements that must be clarified before
empirical testing can follow except for ____.
a. participants will accept the philosophy of a drug-free life
b. participants will be equipped with the life skills necessary to succeed
c. participants will be able to obtain their GED upon completion of the program
d. provides participants with problem-solving skills
Answer: C
11. Program goals represent ____.
a. empirical indicators
b. desired outcomes
c. inputs
d. standards
Answer: B
12. With regard to evaluation designs, which of the following statements is not accurate?
a. variation in the levels of treatment delivered by a program can be a major threat to the
validity of even randomized evaluation studies
b. uncontrolled variation in treatment is equivalent to unreliable measurement of the
independent variable
c. controlled variation in treatment represents a threat to generalizability
d. varying degrees of participation by clients in a program will impact the outcome
Answer: C
13. Randomization may not be appropriate for assignment of people to treatment or programs
for all of the following reasons except ____.
a. practical reasons
b. agency support
c. ethical reasons
d. legal reasons
Answer: B
14. Relating to randomization, which statement is inaccurate?
a. as the number of exceptions to random assignment increases, the statistical equivalence of
the experimental and control groups decreases
b. when exceptions to random assignment are made, no bias enters the selection process
c. randomization in experiments for evaluation purposes are best suited for programs where
exceptions are at a minimum
d. Random assignment of people to receive some especially desirable or punitive treatment
may not be possible for legal, ethical, and practical reasons.
Answer: B
15. If the number of subjects in a program is small, statistical tests can detect only very large
program effects of differences in outcome measures between the two groups. This illustrates
the problem of ____.
a. generalizability
b. statistical conclusion validity
c. maturation
d. reliability
Answer: B
16. Treatment integrity is ____.
a. roughly equivalent to measurement reliability
b. roughly equivalent to measurement validity
c. the difference between the experimental intervention is delivered as intended
d. both reliable and valid
Answer: A
17. If a randomized experiment is used for evaluation purposes, ____.
a. case flow must be adequate to produce enough subjects in both the experimental and the
control groups
b. program staff may minimize the exceptions to randomization
c. special interventions must be consistently applied
d. maintaining the desired level of control over experimental conditions
Answer: A
18. In the home detention studies used in your text as examples of randomized studies, the
authors concluded that ____.
a. randomization allows generalization
b. randomization allows for reliability
c. randomization does not control for variation in treatment integrity and program delivery
d. randomization does control for variation in treatment integrity and program delivery
Answer: C
19. What can be said about the relationship between process evaluations and impact
assessments?
a. process evaluations can be used in interpreting results from an impact assessment
b. process evaluations monitor program implementation but impact assessments do not
answer questions about a program’s effects
c. process evaluation cannot be used to interpret outcomes
d. there is no relationship between impact assessments and process evaluations
Answer: A
20. Which of the following is not correct with respect to process evaluations?
a. information about program implementation can be linked to outcome measures, even when
accompanied by a process evaluation
b. process evaluation aids in interpreting results from impact assessments
c. process evaluations are useful when a researcher is interested in the performance of
specific tasks within a program
d. process evaluations assume that tasks within a program are linked to program outcomes
Answer: A
21. Policy analysis is used to ____.
a. help design alternative courses of action
b. select a policy
c. seek explanations
d. select right employees
Answer: A
22. Policy analysis is ____.
a. infeasible courses of action
b. the generation of information and collection of evidence of the benefits
c. the consequences that would follow the implementation of various courses of action
d. fluid and does not always “start at the beginning and conclude at the end.”
Answer: D
23. Which of the following is not accurate with respect to ethics and the political aspects of
applied research?
a. There is no set of political norms that are agreed upon by all criminal justice researchers.
b. Just as there are codes for ethical conduct for researchers, there are formal codes of
accepted political conduct.
c. Ethics of criminal justice research deals more with the methods used while political issues
are more concerned with the substance and use of the research.
d. There is no formal code of accepted political conduct.
Answer: B
24. Program evaluation differs from policy analysis with respect to ____.
a. when it occurs in the policy process
b. the cost
c. the time
d. the role of the stakeholder
Answer: A
25. Scientific realism is rooted in the principle that:
a. similar interventions can naturally be expected to have different outcomes in different
contexts
b. the success of evaluations depends on the assignment of subjects to experimental and
control groups
c. there be equivalence between experimental and control groups before an intervention
d. treatment integrity must be rigorously guarded
Answer: A
26. Dr. Smith is interested in implementing a program that is designed to reduce the
recidivism rates of property offenders. He selects two state prisons and, with the blessing of
the Department of Corrections, begins the programs in the main state prison for males and a
female institution that is nearby. Which design would be best for Dr. Smith to use?
a. non-equivalent groups
b. randomized study
c. interrupted times-series
d. none of these would work
Answer: A
27. Problem solving is a fundamental tool in problem-oriented policing and involves which
step?
a. Carefully defined specific problems.
b. Conduct in-depth analysis to understand their causes.
c. Undertake broad searcher for solutions to remove these causes and bring about lasting
reductions in problems.
d. All of these answers are correct.
Answer: B
COMPLETION
1. A(n) _______________ is an action taken for the purpose of producing some intended
result.
Answer: policy intervention
2. Policy _______________ refer to what is actually produced.
Answer: outputs
3. The term _______________ refer to the basic question of what a policy action is trying to
achieve.
Answer: impacts
4. Problem analysis takes place in the policymaking stage while ________ studies are
conducted in later stages.
Answer: program evaluation
5. A form of “pre-evaluation” called a(n) _______________ is designed to allow the
researcher to determine whether conditions necessary for conducting an evaluation are
present.
Answer: evaluability assessment
6. A researcher should complete a(n) _______________ that reflects their own understanding
of program goals, elements, and operations of the organization under study.
Answer: program description
7. _______________ are those that have a vested interest in the organization and the outcome
of evaluation research.
Answer: Stakeholders
8. In evaluation studies, the group to whom the program is appropriate is referred to as the
program’s _______________ population.
Answer: Target
Test Bank for Basics of Research Methods for Criminal Justice and Criminology
Michael G. Maxfield, Earl R. Babbie
9781305261105, 9781111346911
