Sources of Crime and Criminal Justice Data

Primary data is collected directly by the researcher to answer a specific question: a new victimisation survey fielded in three cities, or a set of structured interviews with people recently released from prison. The researcher controls the sampling frame, the question wording and the timing, which means the data fits the research question closely, but collecting it is slow and expensive, and it usually covers a narrow slice of time and place.

Secondary data was collected by an agency or an earlier study for a purpose that predates the researcher's question. Most criminology relies on it, because building a new national dataset from scratch is rarely feasible for a single project. A researcher studying sentencing disparity, for instance, is far more likely to obtain court case files or a court administration extract than to interview every judge in a jurisdiction.

The tradeoff is fit versus reach. Secondary sources cover large populations and long time spans that no single research grant could fund directly, but the categories, definitions and missing fields were fixed by whoever built the dataset first, not by the later researcher's hypothesis. A police record that does not distinguish attempted from completed burglary, because operational practice never needed that distinction, cannot be made to distinguish them after the fact.

Most published criminological work sits on a spectrum rather than at either pole: a study might combine a secondary administrative extract with a small primary interview sample added to explain patterns the administrative data cannot interpret on its own. Recognising which parts of a study's evidence are primary and which are secondary is basic groundwork before assessing its conclusions.

Police records are the most widely used administrative source in criminology. They typically include an incident report, an offence classification, and, where relevant, details of an arrest.

In the United States, most police agencies now report incident-level data to the FBI's National Incident-Based Reporting System (NIBRS), which replaced the older summary-based Uniform Crime Reporting (UCR) count as the national collection standard. In England and Wales, the Home Office compiles police-recorded crime data that the Office for National Statistics publishes alongside the independent Crime Survey for England and Wales.

Court records add a second layer: charge sheets, case dispositions, sentencing outcomes and appeal records. These let researchers study case attrition, the drop-off between an initial charge and a final conviction, and sentencing patterns across offence types, courts or demographic groups. Correctional records, covering admissions, time served, disciplinary incidents and release conditions, support research on incarceration trends and reoffending after release.

In India, the National Crime Records Bureau (NCRB) has compiled the annual Crime in India report since 1953, drawing on state police returns under offence categories that now sit under the Bharatiya Nyaya Sanhita, 2023, which came into force on 1 July 2024 and replaced the Indian Penal Code, 1860 as the general criminal code.

Researchers using NCRB series that span the transition need to track which offence definitions changed at the switchover, because a category-to-category comparison across the two codes is not automatic.

The recurring limitation across all three record types is selection. A case only enters a court dataset if police made an arrest and a prosecutor filed a charge; it only enters a correctional dataset if the case ended in a custodial disposition. Each stage filters out cases for reasons that are themselves criminologically interesting, discretion, plea bargaining, resource constraints, so administrative data describes the output of the justice system's decisions at least as much as it describes the underlying offending.

Above individual agency records sits a layer of aggregated national series designed for public reporting and cross-time comparison. The FBI's Crime Data Explorer publishes NIBRS-derived statistics for United States law enforcement agencies; the Office for National Statistics publishes the UK's Crime Survey for England and Wales alongside police-recorded crime; the Australian Bureau of Statistics publishes the annual Recorded Crime - Victims collection; and NCRB's Crime in India report performs the equivalent role for India. Each is built by aggregating agency-level administrative data upward, so it inherits every classification and reporting limitation already present at the source.

Open government data portals have made a growing share of this material directly downloadable rather than locked in periodic PDF reports. Portals such as data.gov (United States), data.gov.uk and India's data.gov.in publish structured datasets, sometimes at a finer geographic resolution than the flagship annual report, letting a researcher build custom local-area or multi-year extracts without submitting a formal data request.

Open portals still carry the same upstream problems as the source records: what is not recorded at the police station is not recorded in the portal either. What they add is transparency about methodology and, in the better-run examples, machine-readable metadata describing exactly how a count was constructed, which is often the only way a researcher can tell whether two years of a series are actually comparable.

National series are also where methodology changes are most visible, and most disruptive to a time series. A change in counting rules, an expanded set of reportable offences, or a shift from summary to incident-based reporting (as with the UCR to NIBRS transition in the United States) can produce a jump or dip in a published rate that reflects the change in method rather than a change in crime. Reading the accompanying technical notes before using a series across a methodology change is not optional.

A cohort study follows a defined group over time rather than sampling a fresh cross-section at each wave. In criminology this usually means either a birth cohort, everyone born in a given place and year, or an offender cohort, everyone who came into contact with the justice system in a given period.

Because the same individuals are tracked repeatedly, cohort data supports questions that a single-wave survey cannot answer directly: at what age does offending typically begin, how long does it persist, and what precedes desistance.

The Cambridge Study in Delinquent Development, begun by Donald West in 1961 and later directed by David Farrington, followed 411 London boys from age eight into adulthood, and remains one of the longest-running criminological cohort studies in the world.

In the United States, Marvin Wolfgang, Robert Figlio and Thorsten Sellin's 1972 study Delinquency in a Birth Cohort tracked boys born in Philadelphia in 1945 and introduced the influential finding that a small share of a cohort accounts for a disproportionate share of its recorded offending, a pattern later described as the chronic offender effect.

Cohort designs carry their own costs. Attrition, cohort members who move, decline further contact or cannot be traced, accumulates with every wave and can bias later findings toward whoever remained easiest to follow. Funding a study across decades is also rare; most cohort studies that have produced durable findings ran for far longer than a typical grant cycle, which is part of why so few exist and why the handful that do are cited so heavily.

Offender cohorts, built from justice-system contact rather than birth records, are cheaper to assemble because the sampling frame already exists in agency data, but they inherit the selection problem described in section 2: an offender cohort defined by arrest already excludes everyone whose offending never resulted in an arrest, which limits what it can say about the causes of offending onset in the wider population.

Victimisation surveys, such as the National Crime Victimization Survey in the United States or the Crime Survey for England and Wales, and self-report offending surveys ask a sample of the population directly about crimes they experienced or committed, independent of whether those incidents were ever reported to police. These surveys were designed to estimate the dark figure of crime, but once fielded, the resulting dataset becomes a reusable secondary resource for questions the original survey team never asked.

A researcher studying fear of crime, for example, might reanalyse several waves of an existing victimisation survey rather than fielding a new one, because the survey already contains relevant questions, a large sample and a known sampling design. This secondary reuse is efficient, but it locks the researcher into whatever variables, wording and sampling frame the original instrument used; a question that was not asked in the original survey cannot be recovered from the dataset later.

Repeated cross-sectional surveys, run on a regular cycle with a fresh sample each time, let researchers track national trends over time even though they do not follow the same individuals, which is the key structural difference from the cohort designs discussed in section 4. Comparing a repeated cross-sectional trend against a police-recorded trend for the same period is one of the standard ways researchers estimate how the dark figure of crime is moving, not just its current size.

Self-report surveys carry a distinct measurement concern: respondents may under-report socially sensitive or illegal behaviour, or occasionally over-report it, and the direction and size of that bias can differ by offence type and by who is asking the question. Researchers reusing an existing self-report dataset should treat its published validation work, where the survey organisation tested question wording against known outcomes, as part of the dataset's documentation, not as optional background reading.

Record linkage combines two or more separate administrative datasets by matching the records that belong to the same individual, using either a shared unique identifier or probabilistic matching on fields such as name, date of birth and address. The technique originates outside criminology, in Halbert Dunn's 1946 proposal for linking vital-statistics and health records, and has since become a standard tool wherever a research question needs a fuller picture than any single agency's records provide on their own.

In criminology, linkage is used to connect justice-system records with health, education, employment or welfare records, which lets researchers ask questions no single-agency dataset can answer: whether a childhood contact with child protective services predicts later justice-system contact, or whether access to employment support after release reduces reoffending. Because each linked dataset was built by a different agency, on a different schedule and with different data standards, linkage projects typically need substantial data-cleaning work before matching can even begin.

Linkage raises privacy and consent questions that a single-agency dataset does not, because a linked record can reveal far more about an individual than any one source alone, and the individual concerned may never have consented to that combination being created. Jurisdictions that support large-scale linkage for research, several university-run data linkage centres in Australia and the United Kingdom are established examples, generally operate under strict de-identification, secure-access and ethics-review requirements precisely because of this added disclosure risk.

Match quality is also a technical limit on its own. Probabilistic linkage produces both false matches, two different people wrongly treated as one, and missed matches, one person's records wrongly treated as belonging to two people, and either error can distort downstream findings. Reporting a linkage study's match rate and validation approach is treated as basic methodological disclosure, not an optional footnote.

Digital trace data refers to records generated automatically as a by-product of online or networked activity: social media posts, mobile phone location logs, online marketplace transactions, or server logs from a platform used to plan or commit an offence. Unlike administrative or survey data, no one designed a collection instrument in advance; the researcher captures activity that was already happening for a different reason, closer in spirit to observing a natural record than to fielding a survey.

The appeal is scale and immediacy. A dataset of millions of social media posts or transactions can be assembled far faster than a national survey, and it can capture behaviour, such as online fraud, cyberbullying or the sale of illicit goods on dark-web marketplaces, that conventional victimisation surveys were never designed to ask about. Cyber-forensics research in particular has drawn on platform and network logs that have no equivalent in traditional crime statistics.

The limits are just as significant. Digital trace datasets are rarely representative of any defined population, because access to the platform that generated them, and willingness to use it in a way that leaves a visible trace, are themselves unevenly distributed.

Platform terms of service, data-protection law and, in the case of law-enforcement-obtained data, evidentiary rules also restrict what a researcher can access and how it can be shared, which makes replication by an independent team harder than with a public statistical series.

Digital trace data is best treated as a complement to administrative and survey sources rather than a replacement for them. It can reveal patterns, in timing, network structure or the language used in fraudulent communication, that would never surface in an aggregate crime count, but establishing that a pattern in a platform log generalises beyond that platform's particular user base requires the same care that any non-random secondary sample demands.

Every source discussed above shares a common vulnerability: the quality of a secondary dataset is fixed by decisions the researcher had no part in. Missing fields, inconsistent classification over time, and gaps where an agency simply did not record a variable are the norm rather than the exception, and they rarely announce themselves in a summary table.

Cross-jurisdiction comparison compounds the problem. Offence definitions differ: what one country's statute treats as a single aggravated offence, another may record as two separate lesser offences. Counting rules differ too, some agencies count every offence within an incident, others count only the most serious offence per incident, which alone can produce very different totals from an identical set of underlying events.

India's 2023 to 2024 transition from the Indian Penal Code and Code of Criminal Procedure to the Bharatiya Nyaya Sanhita and Bharatiya Nagarik Suraksha Sanhita is a concrete example of a definitional break that a naive before-and-after comparison of NCRB figures would need to account for explicitly.

Reporting practices vary further with public trust in the police, awareness of how to report an offence, and cultural norms around disclosing certain crimes, particularly sexual and domestic offences. Two countries with an identical underlying rate of a sensitive offence can show very different recorded rates purely because victims in one context are more willing to report it, which is one reason victimisation surveys are treated as a necessary check on recorded-crime comparisons rather than an optional extra.

Good practice for handling these limits is not exotic: read the technical and methodological notes that accompany a series before comparing it across time or place, treat any comparison spanning a definitional or statutory change as provisional until the break is explicitly modelled, and report missingness rates rather than silently dropping incomplete cases. None of this eliminates the underlying data-quality problem, but it keeps a study's conclusions honest about what the data can actually support.