CHAPTER NINE: PROJECT EVALUATION

One of the primary responsibilities of the public administrator is to assess the cost effectiveness of programs under his or her direction. In an era of "doing more with less," there is little tolerance for ineffective programs. Public administrators use a variety of tools to evaluate effectiveness, often seeking to quantify the inputs and outputs of a particular program. The most familiar example of quantitative techniques is cost benefit analysis (see Gillespie, 1994, 2000; McInnis and Blundell, 2000; Silva, 1998). However, the validity of quantitative cost-benefit analysis declines as the effects of programs become more diffuse, long-range, and of human rather than dollar impact, as they are in crime mapping and analysis. It is not surprising, then, that in our survey, none of the cities in North Carolina or others we interviewed across the country had applied quantitative evaluation methods to the area of GIS in crime analysis. However, we were impressed in interview after interview with direct evidence of efficiency and effectiveness improvements related to measurable gains in law enforcement due to public investment in crime mapping. In terms of policy implications, planning for crime mapping must provide for ongoing program evaluation, and the lack of quantitative cost benefit analysis should not deter funding of GIS technologies for crime analysis.

Project Evaluation's Three Critical Dimensions

Plan for and monitor efficiency

Efficiency gain is defined by delivering the same quantity of output at a lower cost or at the same cost but higher quality (Gillespie, 2000). The Charlotte-Mecklenburg Police Department provides an excellent example of "doing more with less". During a realignment of the police districts in the jurisdiction, interviewee John Couchell noted how GIS improved efficiency:

"We scheduled a meeting with all 12 of our patrol district captains, turned GIS on, and went in and moved neighborhoods around to balance out workload, and reduced the number of man hours it took to redistrict, reapportion police resources using GIS ... GIS enabled us to use a number of different data sources that we hadn't used in the past and we did it in a more effective manner. We were also more efficient because we cut down on the number of hours that it actually took ... we basically sat in a room with all the district captains and in one day, accomplished what took two people six to nine months to accomplish in the past."

The efficiencies in the central office carry over to the level of district police management. As Lieutenant Bjurstrom of the durham, NC, Police Department pointed out:

"One of my jobs is to go through all the reports at the station, every day. So I'm looking at reports, and what this does, by also looking at a map, I can kind of put the information that's going into my brain daily, and see it up on a map, too, so it helps me when I can pretty much tell that I have a pattern developing in an area, that I really have no way of backing up unless I go back and use a paper trail, and that's very hard to do, if I have to go into the files, or the file system, because we file everything by number and not by crime, so it makes it very difficult, whereas this way here, I can take the map and I can use the information that I already have and I can pretty much determine the patterns and the areas."

Another salient aspect of efficiency is that police officers usually work rotating shifts and may have more than one day off between their tours of duty. Unfortunately, just because an officer is off duty does not mean that crime in a beat area stops. Officers returning to duty have to spend time reorienting themselves to the tactical crime situation as it evolved since their last tour of duty. Maps generated by a GIS can make this transition easier. As Sergeant Harold Medlock of the Charlotte-Mecklenburg Police Department said:

"The thing that officers have to remember is that for their eight hours a day, they may know what's going on in that response area, or in that neighborhood, but they're also not working 16 hours a day, and two days off a week, and they may be on vacation, so to get a more global picture of what's going on in a neighborhood or geographic area, the maps may support what we think is going on, but it also tells us, a lot of times, that our assumptions aren't necessarily what we thought they were."

Senior Patrolman Mark McKenzie of the Wilson, NC, Police Department, echoes Sergeant Medlock's observation:

"I believe, since we work four days on and four days off, we lose a little bit while we're off, and when we come on, we can look at crime maps and actually see what's happened in the past week that we've been off, and that helps us to know where we might need to concentrate our patrol a little bit more, if we're having a problem area."

The observations of both of these officers indicate that although no one is tracking man-hours directly, the potentially quantifiable return on the public's investment in GIS for crime analysis units may be found in the savings of time for officers to catch up on events and prepare for the next tour of duty.

Plan for and monitor effectiveness

Gains in effectiveness also result when GIS is used to improve the quality of a current output or to produce an output not previously available. In our survey, many police supervisors were convinced that GIS enabled them to better target crime in their districts by providing a visual representation of their areas of responsibility. When Major Billy White of the Wilson, NC, Police Department was asked if maps provided by his crime analysis officer provided him with new information he replied:

"I guess what the mapping does is, we anticipate the changes going on in a certain area, based on calls for service, and the unit's home address, what area's given out, what officers are responding all the time. With mapping, it's specific about how many we have, what area are they in, and are they more frequent than they were last month or the year before, in comparison to time, month of the year ... And based on calls for service information, and incident information, realigned the four districts to make them even. And we found out, because of our mapping, and because of our crime analysis, that what we thought at one time was the high crime radial, another area was the high crime radial, so we had to increase the manpower in that district, as opposed to the district we thought we were going to have to increase, and it was all due to the crime analysis and to the mapping."

Another area in which GIS is improving the effectiveness is the communication between police departments and the public. Captain Beverly Council of the Durham Police Department noted:

"What also helps with that [mapping], too, as far as getting and knowing where your resources are, you can tell the community, you can be honest with the community, and you can let them know I'm coming into your area, and this is why I'm coming to the area. And the perception from a community perspective is a lot of police out there, I can feel a difference, I can see a difference."

Good communication between a police department and the public may generate other gains in effectiveness. Captain Council went on to note that when a major food franchise decided to relocate close to North Carolina Central University one of the first questions asked by the corporation was "what is the crime rate?"

"He went to Crime Analysis [which houses the mapping operation] and got a report we had been in that area, and the businesses in that area told him that crime had been reduced, and because of that, he brought his business here."

As technology improves, the effectiveness of crime mapping increases. For instance, as more and more law enforcement officers use laptop computers on patrol, they can be given the capacity to communicate with records management databases and CAD mapping systems as officers perform routine stops. Maps can be given to officers via email or the Internet, interfacing with mapping packages such as Arcview. Sergeant Harold Medlock of the Charlotte-Mecklenburg Police Department speculated on this development:

"Our department's a little spoiled. Every officer, every supervisor, every manager, has laptop computers. These laptops can access the LAN system through cars. It would be wonderful to be able to do some of your own mapping ... and I think that's coming, and that's going to provide a tremendous resource to us."

Crime analysis is already an integral part of how police departments provide services to citizens. GIS technologies help communicate the results of crime analysis so police can turn information into action and it allows the analyst to view existing information in new ways. One of the defining strengths of a GIS system is the ability to create new relationships from spatial data and present these relationships in a visual format. Captain Beverly Council of the Durham Police Department spoke to this issue when asked about the investment Durham made in providing GIS to their crime analysis unit:

"Let me put it this way. We have not put enough investment in our GIS and CA unit. They could be giving and doing a lot more for us if they had more financial and staffing input. But right now, what we're getting, we're getting the best of what we've got. It definitely needs more support in this agency, on the training side, as well as on the staffing and technology side. It's worth the money that they're putting into it, but I am definitely going to add they need to be putting more money into it…"

The call by officers such as Captain Council for more investment in crime mapping is in itself further evidence of its effectiveness in law enforcement.

Design ongoing program evaluation

As in other areas of public policy, a comprehensive approach to implementing crime mapping requires an evaluation component. However, because GIS in crime analysis units may be difficult to evaluate using traditional cost benefit analysis, evaluation should emphasize qualitative process analysis. That is, program evaluators should focus on changes in law enforcement processes, such as redistricting, assignment of patrol resources, or community liaison, with a view toward describing whether GIS technology has created time, personnel, or resource savings, or has improved quality without increasing cost. Qualitative process assessment may be based on the use of surveys in a case study methodology among the frontline officers, supervisors, and upper management. Additionally, an evaluator may choose to survey citizen samples in certain areas to determine the spatial impact of more effective resource management by the police as a result of implementing crime mapping.

Durham Police Lieutenant J.M. Bjurstrom gave an example of the type of effectiveness gain an evaluator may look for, in answer to the question, "Whether he or any one else reallocated resources based on information provided by maps?"

"Actually, we've just started to do that, and starting July 5th (2000), we've got a target area, and we noticed from the maps, it's during a three-week time period that we had 40% of all of our crimes happening in one small area, so we've been able to develop a plan to put extra people in there, for a certain time period, and we're also using the times of occurrence of crimes, so we developed a scheduling pattern."

The evaluation instrument for a qualitative process assessment of crime mapping will involve similar questions and responses, not only about allocation of resources, but about all of the functions of crime mapping enumerated in the first chapter of this report. By providing evidence of efficiency and effectiveness gains, the program evaluation effort will provide the justification for the budgetary requests associated with implementing and maintaining crime mapping.

Chapter summary

Though the benefits of using GIS in crime analysis units have yet to be quantified in terms of dollars, there is ample evidence to indicate that using the technology increases both the efficiency and effectiveness of police services to communities. Efficiency gains are evident when departments can reduce the amount of time for administrative tasks such as redistricting police sectors. Effectiveness gains are evident when police allocate resources to fit where crime is actually happening based on GIS data. Lieutenant Michael Wood of the Rochester, NY Police Department succinctly gives an observation on the benefits of using technologies like GIS:

"…when you can start automating, and use your resources most effectively and efficiently, you'rel going to be able to get a lot more work done and, face it, I'm in a medium-sized northeastern city, and we all know, of declining tax base, shrinking budgets, doing more with less, we all know and understand that, and that's where advanced technology can help us. "

When the police are more effective, the public benefits from their investment in GIS technology for crime analysis. Additionally the use of GIS technology can change the public's perception of police effectiveness by using maps to show the actual location of crimes to Community Watch groups and others. Being able to demonstrate to the public the reason for various police strategies is a valuable aid in maintaining cooperation between the community and the police.