Revisiting Crime Rates as a Measure of Crime Prevention Effectiveness: Does the "Crime Drop" Reveal a Policy Effectiveness 'Outcomes' Gap?

Introduction

Crime rates as a measure of crime prevention policy effectiveness is so prevalent as to be seldom questioned, particularly in a high crime environment. An outburst of new scholarship seeking to explain a seemingly significant and sustained ‘crime drop’ in many countries (Brown, 2015a; Farrell, Tilley, & Tseloni, 2014; Sherrell, 2013; Tonry, 2014; Tseloni, Mailley, Farrell, & Tilley, 2010; UNODC, 2014) suggests that discourse on the utility of crime rates as a ubiquitous policy effectiveness and performance measure may be timely, for at least three reasons:

• It may help develop new research questions in the quest for consensus on explanations for the crime drop. Crime rates may not tell the whole story.
• Focusing beyond crime rates as the primary evaluation yardstick might reveal overlooked policy lessons. Policies responsive to falling crime rates, such as to justify police budget cuts (BBC, 2015b; Travis, 2015), may otherwise be implemented without the benefit of a comprehensive explanation of the crime drop, or even whether crime rates remain an adequate measure of policy effectiveness.
• If traditional metrics prove less useful in a new (reduced crime) environment, complementary measures more directly addressing the outcomes ultimately intended by modern crime prevention models might better demonstrate, and help achieve, economic and social benefits and reduced harm from less crime.

The next section outlines New Zealand’s Addressing the Drivers of Crime program as a base for the following three sections to critique crime rate targets as policy outcomes. The next two sections illustrate practical examples in which an ‘outcomes’ focus may help improve policy effectiveness, for better results and policy longevity respectively. The article concludes with practical policy implications and suggestions to help realise crime prevention objectives.

The Addressing the Drivers of Crime program

Advertisement

In 2009, New Zealand’s government embarked on an ambitious “new approach to reducing offending and victimisation" by "working on the causes of crime" (Cabinet, 2009, p. 1; 2010, p. 1). Outlined in an earlier article (Has New Zealand identified the causes of crime?) by late 2014 NZ Police plausibly claimed that a Policing Excellence organisational change program and Prevention First operating strategy (NZ Police, 2012; 2013b, p. 24) "fundamentally transformed the business of policing,.. from a response driven model to prevention-focused policing, [with] a greater focus on preventing crime and reducing victimisation" (NZ Police, 2014b, p. 5).

Target objectives of a 4% increase in prevention tasks, a 13% reduction in overall crime rates and 19% reduction in non-traffic prosecutions were exceeded by the initial June 2014 target date; with results of 5.8%, 20.1% and 41.3% respectively. The Commissioner of Police claimed that this equated to "nearly 90,000 fewer crimes and tens of thousands of fewer victims, greatly reducing pressure on the criminal justice system and freeing-up frontline officer time to focus on crime prevention" (NZ Police, 2014c, p. 4). Police also self-assessed being "well on track" (NZ Police, 2014c, p. 4) to meet new targets under the government's Better Public Services program. These seek by 2017 to reduce violent crime rates by 20%, reoffending and youth crime rates by 25%, and by June 2018 to cut total crime rates 20%.

Crime Rates Do Not Fully Reflect Ultimate Policy Outcomes

Consistent with policy effectiveness and outcomes literature (Bayley & Nixon, 2010; Bourgon, 2007; Buss, Buss, & Hill, 2011; Cook, 2004; Marsh & McConnell, 2010; Matheson & Kwon, 2003; McConnell, 2010, 2011; Mulgan, 2008; OECD, 2002; Productivity Commission, 2013, 2015; Schick, 1996, 2002, 2003; Sherman, 2013), New Zealand’s government regards the setting of high-level targets "an effective way of driving a focus on achieving outcomes" and ensuring "alignment with policy objectives" (Cabinet, 2014, pp. 1-2, 13). Crime rates have a long history as simple, standardised measures of criminal activity, albeit not necessarily “because they provide a better measure but because they provide a familiar measure” (Boivin, 2013, p. 271). Even scholars proposing improvements, such as when ambient-based crime rates may be more accurate (Andresen, 2006; Malleson & Andresen, 2015) often accept standard residential-population based crime rates sufficient for most purposes (Boivin, 2013). At issue in this article is whether crime rates sufficiently reflect ultimate policy outcomes, and whether different measures might usefully complement traditional metrics in a new (reduced) crime environment.

Difficult to identify direct links between policies and crime rates

In criminology literature crime rate targets are often referred to as policy 'outcomes' (eg, Garland, 1996, p. 458), but might more accurately be described as 'outputs' or 'intermediate outcomes' because they do not reflect the ultimate outcomes implicitly intended. In the outcomes literature noted above, the effect or impact of crime prevention policies represent ‘outcome’ objectives (Crawford, 1998, pp. 199, 214; Johnson et al., 2004). Measures may include the number of crimes committed (‘before and after’ comparisons), crimes prevented by the policy intervention (the counterfactual), and the economic and social benefits and reduced harm resulting from less crime. The latter is arguably the ultimate crime prevention policy objective to which other metrics lead, and is used in this article as an ‘outcomes’ exemplar in this context.

It can be difficult to measure or even identify the ultimate policy outcomes sought by crime prevention policies, so causally relevant proxy indicators may be used (Productivity Commission, 2015, pp. 1.10, 11.14), but "having clear goals and targets does not mean that they are actually good goals and targets" (Buss et al., 2011, p. 121). The distinction between clear goals (such as crime rates) and ‘good’ objectives (such as the economic and social benefits from less crime) is especially relevant in the criminal justice context, because it can be challenging to identify direct links between policy initiatives and crime. For example:

a) A more comprehensive evaluation may be necessary because even "successful outcomes in the prevention of crime may not produce reductions in crime rates" (Crawford, 1998, p. 199).

b) Factors not directly related to criminal justice policy initiatives affect crime. Economic conditions (Hagan, 2011, p. 82; Rosenfeld, 2009), technology which 'designs out' crime such as vehicle immobilisers and CCTV (Brown, 2015b; Farrell, Tseloni, Mailley, & Tilley, 2011; Farrell, Tseloni, & Tilley, 2011; Garland, 1996, p. 451), urban design (Butler, 2013; Fennelly & Crowe, 2013; Jeffery, 1971; Newman, 1972) and major event planning (Goldby & Heward, 2013) all affect crime rates.

c) International anti-money laundering norms expressly co-opt the crime prevention capabilities of banks and other private sector agencies (FATF, 2012, 2013). Domestic crime prevention is similarly affected by the extent and effectiveness of third party activities that co-exist in varying degrees of coordination with centralised crime prevention interventions. Building "strong relationships within the community that assist in preventing criminal activity" (Barberini, 2013) has long been part of police craft. Buerger and Mazerolle place these activities within a public third-party policing context to describe the systemic or ad-hoc persuasion of parents, local authorities, property owners and others with influence over targeted individuals and crime 'hot spots' (1998). Devolving crime prevention tasks to third parties, often with police guidance, is a common component in situational crime prevention and problem- and community-oriented policing (Anleu, Mazerolle, & Presser, 2000, p. 72; Barberini, 2013; Gill, 2013; Homel & McGee, 2012; Mazerolle & Ransley, 2004, 2005; Meares, 2006; A. Mills & Codd, 2008; Sarre, 2004; Webster, 2013).

d) Independent private sector innovation also affects crime, sometimes called plural policing (Loader, 2000), or market-based crime prevention or private third-party policing (Anleu et al., 2000). Private security is an early example (Bradley, 2014; Prenzler, Earle, & Sarre, 2009; Prenzler & King, 2002; Sarre & Prenzler, 2000; Shearing & Stenning, 1987; Stenning, 1992). Anleu adds the “regulatory activities” of insurance companies encouraging crime prevention with discounted premiums and deductibles for customers with burglar and car alarms and asset identification systems (2000). Voluntary and community groups also offer support services with crime prevention effects, such as prisoner rehabilitation and reintegration, drug treatment, employment and literacy support programs (A. Mills, 2015; A. Mills, Meek, & Gojkovic, 2012). As Clarke credited new technology for promoting the 'crime science' branch of criminology (2004), emerging trends also involve technology. For example, an online ‘Spotlight’ tool to locate sex traffickers and cut investigation time by 43% (O'Brien, 2015; Thorn, 2015), and 'real-time' risk analysis tools to reduce retail fraud (eStar, 2015) and insurance fraud (ICNZ, 2015; Tibshraeny, 2015). A company with high-profile backers also uses technology to cut high-volume theft-related offending, claimed to represent "a third of all reported crime", currently "the most unsolved of all crimes” because the cost of traditional investigative methods is “prohibitively high" (Auror, 2015). If successful, private sector innovation that cuts a substantial part of “a third of all reported crime” would have a significant impact on crime rates.

Crime rates are affected by many political, economic and social factors. Solving the attribution problem, by establishing causal links between prevention activities and crime rates, is complex, even without the benefit of trend-opposing outcomes, when in a rising crime environment a drop in crime rates may be hailed a policy success with barely a nod to causation principles. In a falling crime setting, however, it is difficult to achieve even the veneer of apparent causality. Robust evaluation is crucial.

Falling crime reveals inadequacies in crime rates performance measures

Advertisement

Whether changing crime rates can be attributed to policy interventions or some combination of prevention initiatives and other factors, the use of crime rate targets is so widespread as to be seldom questioned, particularly in the face of decades of high crime leading into the mid-1990s. Discourse on the continued efficacy of a ubiquitous measure is therefore timely, particularly in light of policy responses to a dramatic and sustained crime drop in many countries. For example:

a) A new status-quo risks becoming associated with justification for policy responses preceding a rigorous consensus understanding of the crime drop. Recent generations may be familiar only with the rising crime era of the second half of the 20th century, but a longer historical perspective reveals a similar period. The Home Office “[made] the most of” an unexpected crime drop in the 1920s to save money by closing prisons (Knepper, 2015, p. 62) and curbing police numbers (Taylor, 1998, 1999). Even as scholars likewise now grapple with the causes and policy implications of the 1990s crime drop, it too has prompted suggestions to cut police budgets in Scotland (Nicholson, 2015) and Canada (Phillips, 2014). Elsewhere this process is further advanced. The English Home Secretary publicly denounced police arguing against proposed budget cuts (BBC, 2015c; Wintour, Perraudin, & Dodd, 2015) as “crying wolf” in the face of falling crime rates (BBC, 2015b). She added that the Chancellor’s subsequent budget cap didn’t let police “off the hook.” The Home Secretary asserted that neither she “nor the public” would “have any sympathy for those who complain about budget cuts” (Travis, 2015). Whatever the ultimate budget impact, the crime rate narrative has changed. Having long relied on crime rates as a measure of effectiveness, English police now argue that such statistics don’t tell the whole story (BBC, 2015a). The Chief Constable responsible for crime statistics acknowledged the challenge of effectively addressing serious and complex crime where “the outcomes of the police response are not easily measured in the current system” (Casciani, 2015, emphasis added).

b) The Queensland Organised Crime Commission of Inquiry extensively investigated “the nature and extent of organised crime,” and “its economic and societal impacts.” The Inquiry found that “extreme legislation” and the “blinkered approach” of a ‘crackdown’ on outlaw motorcycle gangs (representing only 0.52% of crime) meant that authorities “lost visibility” of other serious crime. The Commission also noted ‘outcomes’, as the effects and impact of policy interventions. These included “profound” and “severe” effects of “significant numbers” of children sexually exploited, the “immense” impact of financial crimes “allowed to flourish,” and an estimated 320,000 fraud victims for whom the state had no capacity “to provide an investigative response” (Byrne, 2015, pp. 2, 6, 7, 24, 282).

c) Australia’s Strategic Policy Institute has suggested a “paradigm shift” from traditional approaches to drug offending, asserting that a “principled approach” should “strategically focus on reducing harm to … communities, not on seizing drugs or making arrests” (Coyne, White, & Alvarez, 2015, p. 6, emphasis added). Similarly, calls to remove the inequities of mandatory sentencing (Lee, 2015). The political conversation does not, however, appear to have undergone the suggested paradigm shift. Earlier advice to similar effect was rejected, instead extending hardline criminal justice responses (Courier-Mail, 2015). Australia also continues to advance uncompromising provisions such as mandatory sentencing, three strikes laws, post-sentence preventive detention, ‘control orders’ on children, and automatic citizenship-stripping bypassing judicial overview, notwithstanding legal concerns and evidence that such policies may increase the risks they seek to address (Bullock, 2015; Keyzer & McSherry, 2013; Tibex, 2016; Tulich & Blackbourne, 2015; Zifcak, 2015).

d) In contrast, in an “abrupt public shift in philosophy for… law enforcement officials who have sustained careers based upon tough-on-crime strategies” (Williams, 2015), more than 130 U.S. police chiefs and other prominent law enforcement officials have formed a national alliance (Law Enforcement Leaders, 2015) arguing to cut incarceration rates, mandatory sentencing and zero-tolerance policies. They also seek to eliminate ‘minor’ crimes, such as many narcotics offences, and refocus resources from “arresting and imprisoning people for low-level offences” towards more serious and violent offending (Barned-Smith & Ward, 2015), consistent with scholarly descriptions of the “severity and overbreadth” of “parochial” sentencing laws having “caused enormous unnecessary human suffering” (Tonry, 2015, p. 653). In December 2015, the head of the White House Office of National Drug Control Policy also sought to recalibrate discussion towards ultimate policy ‘outcomes’. He publicly denounced the “failed policies and failed practices" of America’s decades-long ‘war on drugs’. “Blunt force didn't knock out the drug epidemic … We can't arrest and incarcerate addiction out of people… It's … inhumane,.. it's ineffective, and it cost us billions upon billions of dollars” (Botticelli, 2015).

A complex mix of the perceived failure of ‘tough on crime’ policies, a law enforcement focus based on non-validated populist responses allowing serious crime elsewhere to ‘flourish’, and a period of significant change in the criminal environment in many countries arguably exposes flaws in the dominant crime prevention policy effectiveness and performance metric. In the context of modern crime prevention models and a new (reduced crime) environment, policymakers may need new measures to better reflect the ultimate ‘outcomes’ of crime prevention activities, with a more direct line of sight between policy interventions and the economic and social benefits and reduced harm from less crime.

Drug interventions suggest a pathway beyond crime rate ‘output’ measures

Supplementing basic crime rate data with measures of the seriousness of crime has a long history. Crime severity evaluation, initiated by Thurstone (1927) and resurrected by Sellin and Wolfgang’s seminal text (Sellin & Wolfgang, 1964) prompted numerous studies on crime seriousness (Cohen, 1988a, 1988b; O'Connell & Whelan, 1996; Rossi, Waite, Bose, & Berk, 1974), and incorporation in some national statistics, such as the United States, England and Canada (Pease, 1988; Statistics Canada, 2009; Wolfgang, Figlio, Tracy, & Singer, 1985). Crime severity evaluations, however, are often constructed on relative rankings of different crimes based on legal penalties rather than the costs and harm imposed by crime. Due to difficulties comparing various types of severity (eg as between theft and violent assault), ratings based on economic costs offer financial measures as a beguilingly useful common metric. Although they introduce new complications (assessing a monetary value to non-economic harm is far from easy), economic cost evaluations conveniently offer a ready comparison with criminal justice and crime prevention costs, thus enabling cost-benefit analysis (Cohen, 2000).

Much like crime rates themselves, however, such evaluations often measure readily available ‘outputs’ (such as external costs or sentences associated with specific offences) rather than the ultimate ‘outcomes’ of societal benefits and reduced harm from less crime. Similarly, intensive drug enforcement activities may be thought to disrupt and prevent crime, but when subjected to rigorous evaluation may prove to have little sustainable preventive effect (Hicks, 1998; Sherman & Rogan, 1995).

Criminology has begun "grappling with the issue of measuring outcomes and linking outputs to those outcomes", at least in the context of illicit drugs (Maher & Dixon, 1999; Newburn & Elliott, 1998; President of the Council, 1998) and drug seizures (McFadden, 2006, p. 77), yet the measures used to promote policy success may be part of the problem. Traditionally expressed in terms of the quantity and estimated 'street value' of seized drugs and precursor materials, simple 'output' measures fail to reflect the ultimate ‘outcome’ in terms of the effects or impact on society as a result of drugs not reaching their intended market. "In short", said one researcher, "outcomes are the goals or social impacts that the government seeks to achieve through outputs [illicit drug seizures] provided by agencies" (McFadden, 2006, p. 68). Likewise regarding proceeds of crime activity, in which police have traditionally used measures “readily available (eg arrests and forfeitures) rather than measures ... truly appropriate for measuring the impact of proceeds from crime action upon the criminal enterprise and ultimately, through the disruption of crime, on the community" (McFadden, O'Flaherty, Boreham, & Haynes, 2014, p. 3).

Governments and police may set drug seizure targets, and they may be labelled 'outcome’ goals, but arguably the ultimate policy objective implicitly sought is the "improved social, health and economic outcomes in the ... community" associated with reduced consumption and other benefits caused by drug interventions (McFadden, 2006, p. 76). An Australian study estimated A$5.80 of harm reduction benefits for every dollar invested in drug law enforcement (McFadden & Porter, 2011, p. 376). New Zealand research suggested the use of illicit drugs cost NZ$1.31 billion annually, and that drug seizures "between 2000 and 2006 saved NZ$3.67 billion" in harm, including "tangible costs such as crime, lost work output, health service use and other diverted resources, [and] psychological or intangible costs such as reduced quality or length of life" (NZ Police, 2008; 2009, p. 2; Slack et al., 2008).

It is axiomatic that modern policing that successfully interdicts serious criminal activity at an earlier stage helps prevent harm, yet traditional ex-ante crime rate measures may not fully reflect the benefits of preventive policing. NZ Police is in the process of a major organisational transformation, but the core metric of policing effectiveness appears not to have kept pace with the change towards a modern preventive policing model. Crime rate targets may not adequately reflect the ultimate ‘outcomes’ intended by crime prevention, particularly in a falling crime environment.

Crime reduction targets do not fully reflect policy outcomes

"Policing is a practical business and investigators are practical people" (Stelfox & Pease, 2005, p. 204), so a pragmatic conceptual example may help illustrate why crime rate targets do not adequately represent the ultimate outcomes commonly thought. As one target is met, what does it mean? A 20% reduction is, of itself, largely meaningless except as an indicator of the societal benefits represented by that new state. Successive targets (say a further 10%, then another 5% from ever-reducing baselines) seemingly exhibit diminishing returns, yet the impact of successful crime reduction activities, particularly the economic and social benefits from each successive gain, may be at least as significant as the initial target achieved. Even if policymakers and citizens implicitly 'know' the benefits of meeting initial crime rate targets, percentage figures alone (successively reducing as baselines contract) may be deceptive. Crime rate reductions of say 20%, then 10%, then 5% tell us little about policy effectiveness regarding the underlying economic and social benefits; and by seemingly signalling diminishing incremental gains may be misleading.

Particularly as crime rate targets approach, policymakers may wish to consider the benefits of complementary measures of crime prevention effectiveness. The cumulative gains of policy success may be significant, but if not measured the benefits of “better outcomes” and “a safer society” may be obscured by targets expressed as a reducing series of percentage figures. Consistent with high-order objectives expressed as "less harm" in NZ Police's outcomes framework (NZ Police, 2014a) and the "reduced victimisation" of Prevention First, this may involve actually measuring and evaluating the higher order societal benefits of less crime.Continued on Next Page »