gram planning. Nevertheless, to the extent to which program planning is involved (for example, a proposal to provide training of policemen on police-citizen relations would be a program-planning problem), these factors need to be considered. Generally, however, they will be secondary to the fundamental purposes of the service. While measures for these factors are not presented in the illustrative criteria in section III, it may be appropriate for a particular jurisdiction to include such criteria for certain of its analyses.

15. Uncertainties and political considerations are additional evaluation factors

In addition to such nonmonetary criteria as are presented in section III, and the various monetary benefits and costs, other considerations enter into final program decisions. Such factors as the amounts of uncertainty and risks involved (which should be indicated and quantified in the analysis to the extent practicable) and various political considerations may also play important parts in the final decisions. These factors can also be considered evaluation criteria and should not be ignored. Wherever possible they should be discussed, and quantified to the extent practicable, in the analysis. 10

16. Criteria frequently will be difficult to measure

As has already been indicated, it will undoubtedly be extremely difficult to get good historical information on many of these criteria and to make good estimates of the future magnitudes of the criteria for the various program alternatives." In some of these cases, information systems can be feasibly developed to provide improved information in the future. In cases where this appears impossible, it will still be desirable to make crude estimates-based upon judgment if nothing else.

As already noted, at the very least, alternatives can be ranked on each criterion or, a more complex technique, experts can be asked to assign a value to each criterion for each alternative based upon an arbitrary scale (for example, 1 to 10). Public opinion polls, using appropriate sampling techniques, can be used to obtain information on various "intangible" criteria (though there are many difficulties in such polls). Even this information will often be helpful. If even qualitative estimates cannot reasonably be made, substitute criteria will be necessary.

It is to be emphasized that even though an important evaluation criterion resists quantification, this does not mean it should be ignored in the analysis. Relevant qualitative information should be provided; or at the very least the inability to say anything meaningful about the criterion should be clearly pointed out along with its possible implications. The decisionmakers will then at least be alerted to the problem.

10 Note that good analysis in general will not make a decisionmaker's job easier. In fact to the extent that it provides him with additional considerations that previously were hidden, good analysis can actually make his job harder. Good analysis should, however, provide him with considerably improved information on which to base his decisions.

For some criteria, it may be that "reported" data is known to be incomplete. In such cases, estimates of the unreported cases should also be made if this is at all possible. If not, the analysis or the definition of the criteria should at least make clear the omission. Examples of such unreported data are the incidences of various illnesses and of unreported crimes.

17. Intangibles will always be with us

The decisionmaker will inevitably be faced with major intangibles. In addition to the difficulties discussed in "16," certain important aspects of governmental (and perhaps personal) objectives are bound to be omitted from the criteria that are quantified or discussed qualitatively. Since even the type of criteria presented here falls short of indicating ultimate "value" or "utility," and even if all the listed criteria could be satisfactorily quantified, intangibles would still

remain.

For example, though the number of families living in "substandard" dwelling units is a tangible figures, the "value" of reducing this number by various amounts is primarily intangible. Most often the governmental executives will have to make these judgments themselves. There also will be times when such intangible issues should be put before the legislative branch or directly before

the voters.

Program analysis, with the use of such criteria as are contained here, can only aim at improving the relevant information on the issue at hand. It does not need to, nor can it, provide the definitive answers on program selections.

SECTION III. ILLUSTRATIVE LIST OF CRITERIA FOR THE EVALUATION OF PROPOSED PROGRAMS

I. PERSONAL SAFETY 1

Objective: To reduce the amount and effects of external harm to individuals and in general to maintain an atmosphere of personal security from external events.

A. LAW ENFORCEMENT

Objective: To reduce the amount and effects of crime and in general to maintain an atmosphere of personal security from criminal behavior. (To some persons the punishment of criminals may be an important objective in itself as well a means to deter further crimes.)

1. Annual number of offenses for each major class of crime (or reduction from the base in the number of crimes).

2. Crime rates, as for example, the number per 1,000 inhabitants per year, for each major class of crime.

3. Crime rate index that includes all offenses of a particular type (e.g., "crimes of violence" or "crimes against property"), perhaps weighted as to seriousness of each class of offense.

4. Number and percent of populace committing "criminal" acts during the year. (This is a less common way to express the magnitude of the crime problem; it is criminal oriented rather than "crime oriented.")

5. Annual value of property lost (adjusted for price-level changes). This value might also be expressed as a percent of the total property value in the community.

6. An index of overall community "feeling of security" from crime, perhaps based on public opinion polls and/or opinions of experts.

7. Percent of reported crimes cleared by arrest and "assignment of guilt" by a court.

8. Average time between occurrence of a crime and the apprehension of the criminal.2

9. Number of apparently justified complaints of police excesses by private citizens, perhaps as adjudged by the police review board. 10. Number of persons subsequently found to be innocent who were punished and/or simply arrested.

1 Criteria for personal safety are here presented for two subcategories: "Law Enforcement" and "Fire Prevention and Firefighting." Other subcategories could be identified such as "Traffic Safety" (in this paper relevant criteria for traffic issues are included under major program area VII) and "Protection From Natural and Manmade Disasters." The appendix illustrates the particular subcategories that might be included under this, as well as the other, major program areas.

A major purpose of criterion 8 as used in this list is to reflect the psychological reduction in anxiety due to the length of this time period. Note that it is not the purpose of this or any of these criteria to evaluate the efficiency of the police organization.

Notes

(a) Criteria 1 through 6 are criteria for the evaluation of crimeprevention programs. Criteria 7 and 8 are aimed at evaluating crime control after crimes have occurred (i.e., when crime prevention has failed). Criteria 9 and 10 and to some extent 6 aim at the avoidance of law-enforcement practices that themselves have an adverse effect upon personal safety. Criterion 6 and to some extent 8 aim at indicating the presence of a fearful, insecure atmosphere in the locality. (b) Some argue that the primary function of criminal apprehension and punishment is to prevent future crimes; and, therefore, that criteria 7 and 8 would not be sufficiently "end oriented", but rather "means" oriented, and would not be included in the list.

(c) For many analyses it would probably be appropriate to distinguish crime activity by the type of criminal, including such characteristics as age, sex, family income, etc. (juvenile delinquency is an obvious subcategory).

B. FIRE PREVENTION AND FIREFIGHTING

Objective: To reduce the number of fires and loss due to fires. 1. Annual number of fires of various magnitudes (to be defined). 2. Fire rates, for example, number per 10,000 inhabitants per year. 3. Annual dollar value of property loss due to fire (adjusted for price level changes).

4. Annual dollar value of property lost due to fire per $1 million of total property value in the locality.

5. Annual number of persons killed or injured to various degrees of seriousness due to fires.

6. Reduction in number of fires, in injuries, in lives lost, and in dollars of property loss from the base. (These are primarily different forms of criteria 1, 3, and 5 and can be substituted for them.) This reduction might in part be obtained by, for example, drawing inferences from the number of fire code violations (by type) found.3

7. Average time required to put out fires from the time they were first observed, for various classes of fires.

Notes

(a) Criteria 1 through 6 are intended for evaluation of fire prevention programs. Criteria 7 and to some extent 3, 4, and 5 can reflect the results of programs which aim at the control of fires after they have started. Criterion 7 also is a proxy for the anxiety related to duration of fires.

(b) It may be appropriate to distinguish among geographical areas within the jurisdiction.

II. HEALTH

Objective: To provide for the physical and mental health of the citizenry, including reduction of the number, length, and severity of illnesses and disabilities.

3 From current data on the violations found, estimates could be prepared of the number of additional violations that would be found and corrected if more fire-code inspectors were added. However, the more important (that is, the higher level) criterion is not the number of violations found and corrected but the reduction in the number of fires and in the loss of lives and property. To get to this higher level criterion, estimates would have to be made of the consequences of not finding and correcting such violations. This footnote is included to indicate the kinds of inferences that are likely to be needed in program analyses. Similar situations can be identified for many of the other criteria presented in this list.