Skip to content

Face Recognition: NIST FRVT, FBI NGI and Forensic Facial Comparison

The biometric most often pressed into forensic service through CCTV evidence: NIST FRVT evaluation benchmarks (the 1:1 verification + 1:N identification rounds, the 2018 demographic-effects report, the 2022 + 2024 FRVT updates), FBI NGI face module + India CCTNS face module, the ENFSI Best Practice Manual for Facial Image Comparison with its three-method framework (morphological comparison, photo-anthropometric comparison, superimposition), the case-law evolution on face-comparison evidence in the US + UK + India, and the rising challenge of forensic facial-comparison admissibility under Daubert + the 2009 NAS + 2016 PCAST critiques.

By Reviewed by Sourabh

Last updated:

Automated face recognition and forensic facial image comparison are distinct activities. Many automated systems encode face images as numerical embeddings and compare them with enrolled images. NIST’s current evaluation programmes are FRTE for face recognition and FATE for face analysis; older reports use the name FRVT.

Forensic facial image comparison examines similarities and differences in visible facial features. ENFSI’s 2018 Best Practice Manual recommends morphological analysis and stresses image limitations, validation and transparent reporting. Automated candidate retrieval and an evidential examiner opinion require different evaluation and documentation.

A search result is not a forensic conclusion. Interpretation depends on the algorithm and gallery, the image evidence, the examiner’s method and the legal requirements of the jurisdiction.

Key takeaways

  • NIST’s current recognition and analysis programmes are FRTE and FATE; FRVT is the historical name.
  • Demographic error findings depend on the evaluated algorithm, dataset, task, metric and threshold.
  • NGI-IPS must be distinguished from other repositories searched by FBI FACE Services.
  • ENFSI recommends morphological comparison; feature counts and overlays do not establish identity.
  • An incorrect face-recognition lead and inadequate corroboration can together cause serious investigative errors.

A crime-scene face image may be searched automatically to obtain possible candidates, then assessed through further investigation and, where appropriate, documented forensic comparison. Each stage answers a different question and needs its own records and safeguards.

That sequence of events brought into focus the two parallel disciplines that together constitute forensic facial comparison: automated face recognition systems and human forensic facial image comparison. They use different methods, produce different types of output, are evaluated against different error-rate frameworks, and are governed by different evidentiary standards. Both are subject to active scientific critique, regulatory reform, and expanding case law.

This topic covers automated face search, NIST evaluation terminology and metrics, forensic morphological comparison under ENFSI and FISWG guidance, operational database boundaries and limits on evidential interpretation.

By the end of this topic you will be able to:

  • Distinguish automated candidate search from documented forensic facial comparison.
  • Explain 1:1 verification and 1:N searching, including threshold-dependent FMR, FNMR, FPIR and FNIR.
  • Interpret demographic error findings only within their evaluated algorithms, datasets and search conditions.
  • Describe NGI-IPS separately from external repositories searched by FACE Services.
  • Apply systematic morphological comparison and document image limitations, uncertainty and the basis of the opinion.

NIST Evaluation: 1:1 Verification, 1:N Search and Performance Metrics

In a 1:1 verification task, the system compares a claimed identity (a probe face) against a single enrolled template and returns a similarity score. The decision is typically binary (match or non-match) at a defined threshold, with a false match rate (FMR) and false non-match rate (FNMR) traded off by threshold choice.

Law enforcement uses 1:1 verification at border e-gates, building access control, and when a suspect has been identified and their enrolment image needs to be compared against a crime scene image.

In a 1:N task, a probe is compared with a gallery of enrolled identities. Depending on the service, the output may be a ranked candidate list or a threshold-based decision. Gallery size and composition are part of the test conditions and must be reported when interpreting accuracy.

Verification compares a claimed identity with one enrolled identity; identification search compares a probe with a gallery.

Rank-k retrieval asks whether the correct identity appears within the first k returned candidates. At a stated decision threshold, FPIR is the proportion of non-mated searches returning one or more candidates above threshold; FNIR is the proportion of mated searches failing to return the mate above threshold. These search-level metrics differ from pairwise FMR and FNMR.

False match and false non-match describe different errors. Their measured rates depend on the algorithm, images and decision threshold.

NISTIR 8429 (2022), Part 8: Summarizing Demographic Differentials, describes summary measures for demographic effects. It is not a general CCTV performance update. Interpret its measures with the report’s datasets and operating conditions.

NIST has separately studied non-cooperative video recognition through Face in Video Evaluation (FIVE), including a report published in 2017. Video recognition performance depends on capture conditions, algorithm and gallery.

Multiple frames may offer complementary views, but any claimed improvement must identify the tested algorithm, frame-selection or fusion method, dataset and metric. A gain observed in one experiment is not a universal CCTV improvement or evidence of operational adoption.

FBI NGI and India CCTNS: Operational Face Databases

The FBI’s NGI includes the Interstate Photo System (IPS). FACE Services can also search separately held federal and state repositories under the relevant agreements. An aggregate count across accessible sources must not be described as the size of the NGI-IPS gallery.

The FBI describes NGI-IPS as a mugshot repository associated with fingerprint and criminal-history records. Driver’s-licence, passport and visa repositories searched by FACE Services are external sources, not automatically part of that IPS gallery.

NGI-IPS and external photo repositories are distinct. Search access does not make every external image part of the IPS gallery.

Database membership and authorized search access are different matters. Identify the source repository, legal authority and date when discussing which people’s images may be searched.

The FBI describes NGI-IPS outputs as investigative leads rather than positive identifications. Its policy explanation requires manual review and further investigation, and says a candidate response cannot be the sole basis for law-enforcement action. This does not make the result a validated forensic opinion or establish one universal rule for every police service.

Robert Williams’s civil complaint describes an incorrect face-recognition candidate generated through Michigan State Police and a subsequent flawed investigation and photo lineup. The 2024 settlement introduced stronger corroboration and disclosure requirements in Detroit. This case does not establish that the algorithm was correct or that every wrongful arrest had one identical cause.

In March 2020, the Ministry of Home Affairs announced approval for NCRB’s proposed AFRS, intended to use police records for matters including criminals, missing persons and unidentified bodies. NCRB’s archive records a tender cancellation and new procurement activity in 2021. These sources do not establish nationwide commissioning in 2021.

A procurement proposal, a contract and a commissioned national service are different stages. Assess an operational face-recognition system using its specific deployment documentation and relevant performance evidence; a national evaluation or implementation status should not be inferred from a tender alone.

UIDAI collects facial photographs for Aadhaar and reports that a facial modality was added to biometric enrolment deduplication from January 2023. Face images therefore are not used solely for authentication. This does not imply that Aadhaar and police face-search databases are interchangeable or freely shared.

India’s DPDP framework has phased commencement under the November 2025 notification. As of 6 November 2026, the core provisions in the eighteen-month commencement group are not yet operative. Any cross-system sharing analysis must identify applicable Aadhaar and other law, commencement dates, purposes and exemptions.

ENFSI Best Practice Manual: Morphological Comparison and Method Limitations

ENFSI’s Best Practice Manual for Facial Image Comparison, version 01 (January 2018), refers to FISWG and imaging guidance. FISWG is the Facial Identification Scientific Working Group; SWGMAT was a materials-analysis group and should not be substituted for it.

The ENFSI manual recommends morphological analysis. It also reviews approaches including photo-anthropometry and superimposition, highlighting reliability problems rather than presenting them as equally validated alternatives.

Morphological analysis (facial feature comparison) involves systematic comparison of the shape, size, relative position, and qualitative characteristics of anatomical facial features: the morphology of the ear helix and antihelix, the shape of the nasal bridge and tip, the upper and lower lip profile, the distance and shape of the canthi, and the form of the brow ridge.

Morphological comparison documents visible similarities, differences and limitations; agreement in a few illustrated features does not establish identity.

The examiner uses documented feature terminology and records similarities, differences and image limitations. ENFSI’s example checklist has 19 components. Its reporting guidance does not prescribe a universal conclusion scale: the chosen scale should allow support for either competing proposition and an inconclusive outcome.

Photo-anthropometry measures image landmarks and proportions. Ratios can cancel uniform image scaling but remain affected by perspective, head pose, landmark placement and image quality. ENFSI does not recommend photo-anthropometry for comparison using uncontrolled imagery. Simple alignment or geometric correction does not automatically validate it.

Superimposition combines images by overlay, blinking or image portions. Pose and perspective can create misleading apparent agreement; an aligned contour is not proof of identity. ENFSI reviews reliability concerns with these approaches.

Scroll horizontally if needed to see the full table.

ApproachWhat is examinedKey limitation
Morphological analysisVisible facial features and discriminating marksRequires usable imagery, systematic observations and documented interpretation
Photo-anthropometryLandmark measurements and ratiosNot recommended by ENFSI for uncontrolled imagery; perspective and placement affect results
SuperimpositionCombined image contours or portionsAlignment can mislead; apparent overlay agreement is not identification

Admissibility in US, UK and Indian Courts: Daubert, NAS and PCAST

In US jurisdictions applying Daubert, courts assess the relevance and reliability of proposed expert evidence. Testability, peer review, error information and general acceptance are potentially relevant factors in a flexible inquiry, not four cumulative requirements that every method must satisfy identically.

PCAST’s 2016 report addressed validation of specified forensic feature-comparison disciplines. It did not evaluate forensic facial image comparison. Its general emphasis on empirical testing is relevant, but a facial-comparison-specific finding should not be attributed to it.

Facial comparison requires its own empirical evidence concerning examiner performance, task conditions and reporting. A general discussion of validation cannot substitute for such evidence.

Phillips and colleagues (2018) compared facial examiners, reviewers, super-recognizers, control groups and algorithms on a challenging experimental face-pair test. Recent algorithms performed within the human range, with the best above the examiner median. These results are not a universal error rate for low-quality CCTV casework.

The same 2018 study found benefits from combining judgments, including examiner-algorithm fusion. The benefit was demonstrated under that study’s conditions and does not by itself establish admissibility or accuracy for every operational use.

In England and Wales, facial image comparison evidence has been admitted under the framework of R v. Atkins and Atkins (2009), in which the Court of Appeal confirmed that facial comparison evidence from a suitably qualified expert was admissible, provided the examiner disclosed the basis of the comparison and the limitations of the method.

England and Wales guidance stresses the subjective basis of image-comparison opinions, competing propositions, appropriate limitations and independent review. It permits verbal expression of significance, but cautions against statistical probabilities without an appropriate statistical basis. Specific legal requirements and current guidance should be checked for the case.

In India, expert-opinion evidence is addressed by section 39 of the Bharatiya Sakshya Adhiniyam 2023, subject to its application to the proceedings. A facial-comparison report should explain qualifications, methods, source imagery and limitations; the relevance of expert opinion does not itself validate a method.

Admissibility and scientific validation are distinct. Courts assess the proposed evidence under the applicable law; a reliable comparison also needs an appropriate method, suitable images and a reasoned account of observations and limitations.

CCTV Evidence and the Image Quality Problem

The dominant source of facial evidence in criminal investigations is CCTV footage rather than enrolment-quality photography, and it is captured at variable resolution, variable illumination, variable angle, and often with significant compression artefacts. The gap between NIST benchmark performance and operational performance on real CCTV evidence is the central practical challenge in forensic facial comparison.

Face-image suitability depends jointly on resolution, focus, pose, illumination, occlusion, motion and compression. Inter-pupillary distance in pixels can describe image scale, but it is not a universal pass/fail rule for automated recognition or forensic comparison.

Off-axis views can hide features and alter apparent proportions. Meaningful comparison requires assessment of what is actually visible and whether the reference images provide comparable views; one yaw-angle threshold cannot establish suitability for every task.

Suitability depends on the visible information and the intended task. Resolution, pose and other limitations must be assessed together.

Before comparing images, document their source, processing and limitations, and assess whether the features needed for the task are visible. Expertise in image processing and expertise in facial comparison are distinct competencies.

A laboratory should use a documented, validated suitability procedure and record why imagery is usable or inadequate for the proposed task. Quality categories should be defined by that procedure and linked to the information needed for meaningful comparison.

Compression and copying can reduce or alter visible detail. Preserve the native recording where possible, record transformations and inspect the actual face imagery; camera specifications or codec names alone do not establish forensic suitability.

  1. Acquire and preserve the CCTV source
    Obtain a forensic copy of the original digital video file from the recording device. Do not work from a re-recorded or compressed copy. Document the chain of custody, the device make and model, the compression codec and bitrate, and the date-time synchronisation status.
  2. Measure image quality
    Extract suitable frames and document resolution, focus, pose, illumination, occlusion and compression. Use task-specific validated suitability criteria rather than universal pixel boundaries.
  3. Assess comparability
    Assess whether viewpoints, camera geometry and expressions permit meaningful feature comparison. Record processing and avoid assuming that alignment recovers obscured or missing information.
  4. Apply validated comparison procedure
    Use a validated morphological comparison procedure appropriate to the available features. Do not select photo-anthropometry or overlays as identification methods merely because a pose threshold is met.
  5. Record each feature finding
    Record visible feature similarities, differences and limitations with standardized terminology. Feature counts alone do not determine evidential weight.
  6. Formulate and report conclusion
    Report the observations, competing propositions where applicable, uncertainty and limitations under the applicable validated reporting procedure. Do not present an unsupported numerical likelihood ratio.
Key terms
NIST FRTE / FATE (formerly FRVT)
NIST’s face technology evaluation programmes: FRTE evaluates recognition and FATE evaluates analysis tasks. Historical reports use the name Face Recognition Vendor Test (FRVT).
False positive identification rate (FPIR)
At a stated threshold, the proportion of non-mated 1:N searches returning one or more candidates above threshold. It differs from pairwise false match rate and rank-k retrieval.
Deep convolutional neural network (DCNN) embedding
A numerical image representation produced by a deep convolutional neural network and compared using a similarity measure. Modern face-recognition architectures and representations vary.
NGI face component
The Interstate Photo System within NGI supports face-search leads. It is distinct from external repositories that FBI FACE Services may search under agreements.
AFRS (India)
NCRB’s proposed Automated Facial Recognition System, approved for implementation in 2020. Approval and procurement records do not by themselves establish nationwide commissioning.
ENFSI BPM
ENFSI’s 2018 Best Practice Manual for Facial Image Comparison recommends morphological analysis, validated procedures, attention to image limitations and transparent reporting.
Morphological analysis
The systematic comparison of qualitative facial feature shapes from a standardised feature list; the primary method in ENFSI BPM facial image comparison, applicable across a wider range of image quality conditions than photo-anthropometry.
Photo-anthropometry
Measurement of facial image landmarks and ratios. Perspective, pose, expression and landmark placement affect results; ENFSI does not recommend its use for comparison of uncontrolled imagery.
IPD pixels
Distance between pupil centres measured in image pixels, describing image scale. It does not by itself establish suitability or a universal recognition threshold.
Verbal likelihood ratio (VLR)
Verbal expressions can describe the strength of support for competing propositions under a defined reporting procedure. They should not be treated as a measured numerical likelihood ratio without calibration or as a universal ENFSI scale.
Practice
Question 1 of 5· 0 answered

How should NISTIR 8280 (2019) demographic error findings be interpreted?

Can face recognition tell identical twins apart?
Twin faces can be difficult for automated and human comparison. NIST’s twins evaluations document substantially higher false match rates for many tested algorithms than for unrelated comparisons at the same threshold. Results depend on algorithm, capture conditions and image evidence; do not infer a reliable individual identification or court acceptance from a general benchmark.
Does wearing a face mask defeat face recognition?
A mask hides information and can reduce recognition accuracy, but its effect varies. NIST’s 2020 study used digitally applied masks and 1:1 algorithms; the best algorithms could have masked-image failure rates around 5%, while many others failed more often. A later 2020 report tested newer submissions without knowing whether each was designed for masks. These results are not universal rates for physical masks, gallery searches or human comparison of CCTV.

Sources

  1. 1.Best Practice Manual for Facial Image Comparison, 2018 · ENFSIMorphological comparison and method limitations
  2. 2.Face Technology Evaluations · NISTFRTE and FATE scope
  3. 3.Law Enforcement’s Use of Facial Recognition Technology · FBINGI-IPS and FACE Services repositories
  4. 4.Facial Comparison Overview and Methodology Guidelines, 2.1 · FISWGMorphological comparison and limits on overlays

Test yourself on Fingerprint Sciences with free, timed mocks.

Practice Fingerprint Sciences questions

Found this useful? Pass it along.