Measuring organised crime: challenges and solutions for collecting data on armed illicit groups
May 2022
Briefing Note 03
Professor Christopher Blattman, Innovations for Poverty Action (IPA) & University of Chicago
Dr Benjamin Lessing, University of Chicago
Juan Pablo Mesa-Mejía, Innovations for Poverty Action (IPA)
Dr Santiago Tobón, Innovations for Poverty Action (IPA) & Universidad of EAFIT
SOC ACE Project: Developing government information and accountability systems for combating serious organized crime: Medellín demonstration project
PUBLICATION SUMMARY
Organised criminal activities, by their nature, are hard to measure. Administrative data are often missing, problematic, or misleading. Moreover, organised criminal activities are under-reported, and under-reporting rates may be greatest where gangs are strongest. Researchers hoping to quantify organised crime systematically face daunting challenges.
Collecting information on organised crime is inherently a slow process of cautious trial and error. It will vary from city to city, and typically within a city as well. Dozens of qualitative and quantitative researchers have shown that this can be done with care, ethically, and with adequate protection for human subjects. What they all have in common is that they commit themselves to a place, and they all take their time.
While there are risks, the benefits can be enormous. The information these investigators collect is often rare and invaluable. Officials and policymakers commonly have little insight into criminal organisations, with terrible consequences for policy, be it inaction, mediocrity, or adverse and unintended consequences. Here we draw on our experience in Colombia, Brazil, and Liberia of collecting systematic data on illicit activities and armed groups, in order to share our learning with other researchers or organisations that fund research in this area, who may find this useful for their own research. We address: first steps before asking questions, common challenges and solutions, and alternative sources.
Our work thus far emphasises the relevance of deep qualitative work to identify local partners; the need for intense piloting of survey instruments and a close oversight of survey firms, ranging from how they hire enumerators to how they plan and implement field work; the power of using survey experiments to mitigate and measure measurement error; and the relevance of cross-validating findings with complementary data sources.
