California Cities Develop Recommendations for Evaluating Gang Initiatives

by Andrew O. Moore

In recent years, cities have taken bold steps to advance comprehensive local gang reduction strategies that blend prevention, intervention and enforcement. But how do municipal leaders know if these efforts are working? While evaluators can sometimes isolate the impact of program-level interventions, it can be challenging to understand the impact of comprehensive strategies on community safety.

Representatives of five California cities gathered with research and evaluation partners this month to develop "recommendations from the field" for evaluating comprehensive gang prevention and reduction initiatives. Each of the cities is a member of the California Cities Gang Prevention Network, an initiative launched in 2007 by NLC's Institute for Youth, Education and Families and the National Council on Crime and Delinquency (NCCD).

The California Endowment, which along with The California Wellness Foundation supports the Network, hosted the meeting at its Oakland office.

Mayor's offices from San Bernardino and Los Angeles sent representatives, and San José, Santa Rosa and Oakland sent senior city staff. Evaluators included professors at two California State University branches and leaders from private evaluation firms, Community Crime Prevention Associates and Resource Development Associates.

Alex Busansky, president of NCCD, described the challenge of the meeting as "how do you measure, articulate and quantify success."

NLC senior consultant Jack Calhoun said the meeting would explore "what we know, think we know, don't know and should know."

In the course of the meeting, each participating city described its evaluation processes and findings to date via the case study method. Whereas all shared the goal of evaluating the success of comprehensive initiatives encompassing many programs, most evaluations to date have focused on individual programs. All five cities have made substantial progress in tracking outcomes and maintain an ongoing focus on documenting gains in their gang prevention efforts:

For instance, San Bernardino has documented a 40 percent reduction in violent crime, and enlisted California State University-San Bernardino to conduct surveys that showed an increasing sense of safety among residents in an area targeted for services. To date, the city has not secured the resources to update the community safety survey, nor to undertake a full-scale evaluation of its Operation Phoenix initiative.

Oakland evaluates its Measure Y violence prevention and community policing initiative to understand whether programs funded by the Measure Y parcel tax and parking surcharge affect school outcomes, recidivism, criminal activity, truancy and attendance, employment and reductions in shootings and homicides.

Notably, Oakland cross-analyzes data from the Oakland Unified School District, Alameda County juvenile and adult probation, the California Department of Corrections and Rehabilitation, Oakland Police Department and service providers.

Los Angeles commissioned a report by the Urban Institute, which found that gang crime decreased more rapidly in the city's 12 Gang Reduction and Youth Development zones compared with other areas of the city. Based on ongoing assessment using a new Youth Services Eligibility Tool, the report identified statistically significant decreases in youth risk factors and gang involvement.

Santa Rosa measures efforts and results of its Community Helping Our Indispensable Children Excel (CHOICE) initiative. A yearly report identified several desirable trends - higher academic performance scores, lower school suspension rates and juvenile arrests - and flagged negative trends such as dropping graduation rates and increasing truancy rates.

San José, along with Santa Rosa, seeks to understand progress in a continuous quality improvement framework. The theory of change/logic model evaluation method provides a means for the city to document change, build capacity and a culture of learning, and document efficiency and effectiveness to attract additional resources.

Participants also reflected on the frequent call for government agencies to fund only "evidence-based practices" in the current tight budget era, noting that most interventions for which evaluators have compiled sufficient evidence occur at the program level, not the comprehensive level. Los Angeles' evaluation experience to date has prompted the city to clarify its conceptual framework, two main goals and related strategies as a platform for understanding impact.

Turning to the development of recommendations for advancing evaluations of comprehensive initiatives, participants explored options such as:

  • Using mixed quantitative and qualitative methods;
  • Forming comparison groups between neighborhoods or similar cities; and
  • Gathering data to answer questions about involvement and effects at a variety of levels - community, family, youth, service delivery agencies and collaborations.

Final recommendations will appear in a forthcoming California Cities Gang Prevention Network publication.

The evaluation convening comes on the heels of the network's publication of three new municipal action guides focused on heightening the role of prevention, reducing violence through reentry strategies, and evaluating comprehensive strategies. All guides are available at www.nlc.org/iyef and www.ccgpn.org.

Details: To learn more about the California Cities Gang Prevention Network, please contact Andrew Moore at (215) 848-6910 or moore@nlc.org.