Revisiting Crime Rates as a Measure of Crime Prevention Effectiveness: Does the "Crime Drop" Reveal a Policy Effectiveness 'Outcomes' Gap?

Translating Policy Effectiveness Into Harm-Focused Policing

A developing area of research holds that the value of focusing directly on harm extends beyond the financial evaluation of policy success. Translating policy interventions into implementation, for example with harm-focused policing, helps improve effectiveness by enabling better outcomes. This is because policy interventions and operational decisions based directly on outcome goals can be better tailored for optimal policy effectiveness than those based on proxy outcome indicators such as crime rates; with focus and resources directed specifically towards activities with a more direct line of sight to ultimate policy goals.

There is some evidence of agencies seeking to connect resource allocations with primary crime prevention ‘outcome’ objectives such as the economic and social benefits and reduced harm impacts of crime prevention initiatives. At a supra-national level, for example, Europol's Serious and Organised Threat Assessment considers the harm resulting from crime to help improve decision-making and policymaking, and for more focused prioritisation of resources (Europol, 2013, pp. 42, 44; Paoli, 2014). Similarly at a national level, the UK offers an example of crime prevention policymaking that sets out "the threats and harms which have a national dimension and provides a framework for police forces … [to] prioritise the response against those groups causing the greatest harm" (National Crime Agency, 2015, pp. 9, 12). Australia’s Costs of Serious and Organised Crime report also measured “some of the financial costs of harms” explicitly to “inform decisions about policy and responses,… [enabling] Australia to target resources to counter the greatest harms” (ACC, 2015, p. 2).

Ratcliffe suggests that "harm-focused policing aims to inform policing priorities by weighing the harms of criminality [and focusing] police resources in furtherance of both crime and harm reduction" (2015a, p. 3). In a practical example, he illustrated side-by-side spatial 'heat maps' of violent crime density and harm. While the charts were similar (and the main hot-spot identical), the significance of some areas changed:

...when examining overall harm Area B increases in significance while Area C decreases in significance. Area B is indicative of a place that experiences significant harm from crime, yet may not have received attention had the focus remained solely on violence. Harmful places are not just areas where violence occurs, but where a plethora of detrimental behaviors damage community life (2015a, p. 5).

Advertisement

Explicitly considering the adverse economic and social impacts of crime (Paoli & Greenfield, 2013), harm-focused policing linked to existing deployment and performance measures provides a more nuanced crime hot-spot analysis than traditional intelligence-led policing techniques alone: on the most harmful places, the most harmful offences and the most harmful offenders; and better reflecting community concerns and crime prevention ‘outcome’ goals.

A body of literature offers frameworks for assessing and evaluating harm "as distinct from the perceived seriousness or costs of crime" (Greenfield & Paoli, 2013, p. 864; Loader, 2014; McCollister et al., 2010; Paoli, Greenfield, & Zoutendijk, 2013; Ratcliffe, 2015b; von Hirsch & Jareborg, 1991). The field is still developing (Greenfield & Paoli, 2013), but a focus on the consequences of crime, within a contemporary program evaluation model (Maxfield & Babbie, 2015), offers proactive police leaders and policymakers opportunities to lead a robust evidence-based approach demonstrably to cut the harms caused by crime (Prenzler, 2014, p. 16).

A strong evidence base enabling more efficient strategies and better outcomes meeting policy intentions, such as harm-focused policing with appropriate ‘outcome’ metrics, may help extend policy effectiveness, but for policy longevity the evidence base also needs to be sufficiently robust to withstand scrutiny, as addressed in the following section.

Robust Evaluation Withstands Scrutiny

At the broader policy-making level in crime prevention policy effectiveness lies another risk associated with 'common sense' decision-making. Crime rate reductions resulting in "nearly 90,000 fewer crimes and tens of thousands of fewer victims" (NZ Police, 2014c, p. 4) and a "30 percent drop in crime" (Adams, 2015a) is easily commended on common sense grounds. For any policy intervention, however, such claims should also be capable of withstanding scrutiny. 'Common sense' declarations that rest at unsupported assertion risk being dismissed by equally simple 'common sense' counter-arguments.

For example, at the macro level, neither of the above success claims may have much to do with policy interventions if they simply reflect falling crime rates throughout the Western world (Farrell et al., 2014; Tonry, 2014). At the micro level, assertions of "90,000 fewer crimes", without more, also leaves open the prospect of other explanations, such as crime displacement and diffusion effects.

Even aside from potential displacement effects, using basic crime statistics for illustrative purposes, New Zealand's drop in the number of recorded offences and crime rates accompanied an increase in both measures for some serious crimes. We might ask whether an additional 949 serious assaults resulting in injury is in effect offset by 1,712 fewer theft offences (NZ Police, 2015b). Clearly, no-one would suggest a direct comparison is appropriate, and NZ Police actively target more serious offences that “cause the most harm to our communities” (Nicholls, 2014).

More accurate harm-focused analysis would, however, allow police to better demonstrate the effectiveness of such efforts. Crime rate indices as performance measures create an obvious incentive to clear 'volume' offending that has a disproportionate effect on overall unweighted crime rates. This inherent disincentive is of course mitigated by the common practice of multi-layered crime rate targets for specific offences, and police target serious offences in any event (as noted above), but the underlying conflict between police professionalism and simplistic performance measures remains. Professionals 'doing the right thing' by focusing on more harmful crime even when it has less impact on overall crime rates is an indicator that crime rate targets as policy goals may not fully represent the harm-reduction outcomes ultimately intended by crime prevention policies.

These observations apply generally, whichever ‘crime rate’ metric is used, whether based on police-reported crime or a broader measure of crime experiences. New Zealand's Crime and Safety Survey (NZCASS) released in late 2015 revealed a 30 percent reduction in crime experiences between 2008 and 2013, but this finding is insufficient to "prove" the success of the government’s crime prevention program, as claimed by the Minister of Justice (Adams, 2015a; Sachdeva, 2015). The crime prevention program likely had some effect. The impact may even have been significant, but other factors might have had as much effect, or more. The report itself notes that the NZCASS crime drop mirrors police-recorded crime reductions in the same period (MOJ, 2015, p. 117).

Other reports confirm New Zealand’s falling crime rates from early- to mid-1990s (Aebi & Linde, 2010; Johnston, 2011; Mayhew, 2012; NZ Herald, 2014; Sachdeva, 2015; Sherrell, 2013; Statistics New Zealand, 2001, 2006). Similar trends appear generally in much the same period in many other countries (Barker, 2010; Blumstein & Wallman, 2006; Casciani, 2015; Farrell et al., 2014; Ouimet, 2002; The Economist, 2013; Tonry, 2005, 2014; Tonry & Farrington, 2005; van Dijk, Tseloni, & Farrell, 2012; Zimring, 2006). A plausible explanation, therefore, is that New Zealand’s crime drop may have had as much to do with external factors “that changed more or less universally” (Tseloni et al., 2010, p. 389) as with any direct causal impact from its domestic crime prevention program.

The Police Commissioner's response was more measured, welcoming “good news” without claiming credit (Bush, 2015a). Police presumably had insufficient evidence from which properly to interpret effectiveness. On the Maryland Scientific Methods Scale of methodological quality, for example, simple 'before and after' observations do not reach minimum interpretable standards because it fails to disentangle plausible alternatives, such as existing trends (Sherman et al., 2002). Absent rigorous evaluation, the lack of sound reasons for global crime reductions renders policy explanations tenuous at best (Farrell, Tilley, Tseloni, & Mailley, 2010; van Dijk, 2006).

Advertisement

It is unremarkable that politicians claiming success follows good news1. Falling crime rates have long attracted policy effectiveness claims, today much the same as in response to ‘the English miracle’ of the 1920s crime drop (Taylor, 1999, p. 121). However, the extent to which crime reduction is attributable to policy intervention, then as now, can only properly be demonstrated with robust evidence separating the effects of other factors. Particularly in a falling crime environment, not conducting rigorous analysis unnecessarily leaves a methodological ‘hostage to fortune’ exposed to the simplest ‘common sense’ rebuttals. Even when policy success claims are justified, failing to support them with rigorous analysis unpacking other relevant factors exposes them to criticism that risks curtailing the longevity even of successful crime reduction policies.Continued on Next Page »