Face Recognition: NIST FRVT, FBI NGI and Forensic Facial Comparison
The biometric most often pressed into forensic service through CCTV evidence: NIST FRVT evaluation benchmarks (the 1:1 verification + 1:N identification rounds, the 2018 demographic-effects report, the 2022 + 2024 FRVT updates), FBI NGI face module + India CCTNS face module, the ENFSI Best Practice Manual for Facial Image Comparison with its three-method framework (morphological comparison, photo-anthropometric comparison, superimposition), the case-law evolution on face-comparison evidence in the US + UK + India, and the rising challenge of forensic facial-comparison admissibility under Daubert + the 2009 NAS + 2016 PCAST critiques.
Automated face recognition and forensic facial image comparison are distinct activities. Many automated systems encode face images as numerical embeddings and compare them with enrolled images. NIST’s current evaluation programmes are FRTE for face recognition and FATE for face analysis; older reports use the name FRVT.
Forensic facial image comparison examines similarities and differences in visible facial features. ENFSI’s 2018 Best Practice Manual recommends morphological analysis and stresses image limitations, validation and transparent reporting. Automated candidate retrieval and an evidential examiner opinion require different evaluation and documentation.
A search result is not a forensic conclusion. Interpretation depends on the algorithm and gallery, the image evidence, the examiner’s method and the legal requirements of the jurisdiction.
Key takeaways
- NIST’s current recognition and analysis programmes are FRTE and FATE; FRVT is the historical name.
- Demographic error findings depend on the evaluated algorithm, dataset, task, metric and threshold.
- NGI-IPS must be distinguished from other repositories searched by FBI FACE Services.
- ENFSI recommends morphological comparison; feature counts and overlays do not establish identity.
- An incorrect face-recognition lead and inadequate corroboration can together cause serious investigative errors.
A crime-scene face image may be searched automatically to obtain possible candidates, then assessed through further investigation and, where appropriate, documented forensic comparison. Each stage answers a different question and needs its own records and safeguards.
That sequence of events brought into focus the two parallel disciplines that together constitute forensic facial comparison: automated face recognition systems and human forensic facial image comparison. They use different methods, produce different types of output, are evaluated against different error-rate frameworks, and are governed by different evidentiary standards. Both are subject to active scientific critique, regulatory reform, and expanding case law.
This topic covers automated face search, NIST evaluation terminology and metrics, forensic morphological comparison under ENFSI and FISWG guidance, operational database boundaries and limits on evidential interpretation.
By the end of this topic you will be able to:
- Distinguish automated candidate search from documented forensic facial comparison.
- Explain 1:1 verification and 1:N searching, including threshold-dependent FMR, FNMR, FPIR and FNIR.
- Interpret demographic error findings only within their evaluated algorithms, datasets and search conditions.
- Describe NGI-IPS separately from external repositories searched by FACE Services.
- Apply systematic morphological comparison and document image limitations, uncertainty and the basis of the opinion.
How Modern Face Recognition Works: Deep Learning and Gallery Search
Many modern face-recognition systems use neural networks to convert an image into a numerical feature vector or embedding. Similarity scores compare these representations. Training aims to accommodate differences in pose, lighting, age and expression, but those factors can still affect performance; no embedding guarantees separation of every identity.
The embedding is computed from a normalised face region. Before embedding, the system must:
- Detect the face in the full image.
- Locate facial landmarks (eye centres, nose tip, mouth corners).
- Align the face to a canonical pose using those landmarks.
- Normalise illumination.
Failures at any preprocessing step propagate into embedding errors: a face detected at the wrong scale, a landmark placed incorrectly on a face in three-quarter view, or extreme backlighting will each produce an unreliable embedding regardless of the underlying network quality.
NIST’s Face Recognition Technology Evaluation (FRTE) evaluates recognition algorithms, including 1:1 verification and 1:N identification. Face Analysis Technology Evaluation (FATE) includes tracks such as image quality and presentation-attack detection. Historical Face Recognition Vendor Test reports retain their FRVT titles.
The NIST FRVT demographic analysis published in December 2019 as NISTIR 8280 examined 189 algorithms from 99 developers across demographic groups defined by sex, age, and country of birth.
NISTIR 8280 (2019) found demographic differentials that varied across algorithms, datasets and tasks. Its findings must not be reduced to a universal ranking of demographic groups: 1:1 and 1:N errors can differ, and image quality and operating thresholds matter.
NIST Evaluation: 1:1 Verification, 1:N Search and Performance Metrics
In a 1:1 verification task, the system compares a claimed identity (a probe face) against a single enrolled template and returns a similarity score. The decision is typically binary (match or non-match) at a defined threshold, with a false match rate (FMR) and false non-match rate (FNMR) traded off by threshold choice.
Law enforcement uses 1:1 verification at border e-gates, building access control, and when a suspect has been identified and their enrolment image needs to be compared against a crime scene image.
In a 1:N task, a probe is compared with a gallery of enrolled identities. Depending on the service, the output may be a ranked candidate list or a threshold-based decision. Gallery size and composition are part of the test conditions and must be reported when interpreting accuracy.
Rank-k retrieval asks whether the correct identity appears within the first k returned candidates. At a stated decision threshold, FPIR is the proportion of non-mated searches returning one or more candidates above threshold; FNIR is the proportion of mated searches failing to return the mate above threshold. These search-level metrics differ from pairwise FMR and FNMR.
NISTIR 8429 (2022), Part 8: Summarizing Demographic Differentials, describes summary measures for demographic effects. It is not a general CCTV performance update. Interpret its measures with the report’s datasets and operating conditions.
NIST has separately studied non-cooperative video recognition through Face in Video Evaluation (FIVE), including a report published in 2017. Video recognition performance depends on capture conditions, algorithm and gallery.
Multiple frames may offer complementary views, but any claimed improvement must identify the tested algorithm, frame-selection or fusion method, dataset and metric. A gain observed in one experiment is not a universal CCTV improvement or evidence of operational adoption.
FBI NGI and India CCTNS: Operational Face Databases
The FBI’s NGI includes the Interstate Photo System (IPS). FACE Services can also search separately held federal and state repositories under the relevant agreements. An aggregate count across accessible sources must not be described as the size of the NGI-IPS gallery.
The FBI describes NGI-IPS as a mugshot repository associated with fingerprint and criminal-history records. Driver’s-licence, passport and visa repositories searched by FACE Services are external sources, not automatically part of that IPS gallery.
Database membership and authorized search access are different matters. Identify the source repository, legal authority and date when discussing which people’s images may be searched.
The FBI describes NGI-IPS outputs as investigative leads rather than positive identifications. Its policy explanation requires manual review and further investigation, and says a candidate response cannot be the sole basis for law-enforcement action. This does not make the result a validated forensic opinion or establish one universal rule for every police service.
Robert Williams’s civil complaint describes an incorrect face-recognition candidate generated through Michigan State Police and a subsequent flawed investigation and photo lineup. The 2024 settlement introduced stronger corroboration and disclosure requirements in Detroit. This case does not establish that the algorithm was correct or that every wrongful arrest had one identical cause.
In March 2020, the Ministry of Home Affairs announced approval for NCRB’s proposed AFRS, intended to use police records for matters including criminals, missing persons and unidentified bodies. NCRB’s archive records a tender cancellation and new procurement activity in 2021. These sources do not establish nationwide commissioning in 2021.
A procurement proposal, a contract and a commissioned national service are different stages. Assess an operational face-recognition system using its specific deployment documentation and relevant performance evidence; a national evaluation or implementation status should not be inferred from a tender alone.
UIDAI collects facial photographs for Aadhaar and reports that a facial modality was added to biometric enrolment deduplication from January 2023. Face images therefore are not used solely for authentication. This does not imply that Aadhaar and police face-search databases are interchangeable or freely shared.
India’s DPDP framework has phased commencement under the November 2025 notification. As of 6 November 2026, the core provisions in the eighteen-month commencement group are not yet operative. Any cross-system sharing analysis must identify applicable Aadhaar and other law, commencement dates, purposes and exemptions.
ENFSI Best Practice Manual: Morphological Comparison and Method Limitations
ENFSI’s Best Practice Manual for Facial Image Comparison, version 01 (January 2018), refers to FISWG and imaging guidance. FISWG is the Facial Identification Scientific Working Group; SWGMAT was a materials-analysis group and should not be substituted for it.
The ENFSI manual recommends morphological analysis. It also reviews approaches including photo-anthropometry and superimposition, highlighting reliability problems rather than presenting them as equally validated alternatives.
Morphological analysis (facial feature comparison) involves systematic comparison of the shape, size, relative position, and qualitative characteristics of anatomical facial features: the morphology of the ear helix and antihelix, the shape of the nasal bridge and tip, the upper and lower lip profile, the distance and shape of the canthi, and the form of the brow ridge.
The examiner uses documented feature terminology and records similarities, differences and image limitations. ENFSI’s example checklist has 19 components. Its reporting guidance does not prescribe a universal conclusion scale: the chosen scale should allow support for either competing proposition and an inconclusive outcome.
Photo-anthropometry measures image landmarks and proportions. Ratios can cancel uniform image scaling but remain affected by perspective, head pose, landmark placement and image quality. ENFSI does not recommend photo-anthropometry for comparison using uncontrolled imagery. Simple alignment or geometric correction does not automatically validate it.
Superimposition combines images by overlay, blinking or image portions. Pose and perspective can create misleading apparent agreement; an aligned contour is not proof of identity. ENFSI reviews reliability concerns with these approaches.
Scroll horizontally if needed to see the full table.
Admissibility in US, UK and Indian Courts: Daubert, NAS and PCAST
In US jurisdictions applying Daubert, courts assess the relevance and reliability of proposed expert evidence. Testability, peer review, error information and general acceptance are potentially relevant factors in a flexible inquiry, not four cumulative requirements that every method must satisfy identically.
PCAST’s 2016 report addressed validation of specified forensic feature-comparison disciplines. It did not evaluate forensic facial image comparison. Its general emphasis on empirical testing is relevant, but a facial-comparison-specific finding should not be attributed to it.
Facial comparison requires its own empirical evidence concerning examiner performance, task conditions and reporting. A general discussion of validation cannot substitute for such evidence.
Phillips and colleagues (2018) compared facial examiners, reviewers, super-recognizers, control groups and algorithms on a challenging experimental face-pair test. Recent algorithms performed within the human range, with the best above the examiner median. These results are not a universal error rate for low-quality CCTV casework.
The same 2018 study found benefits from combining judgments, including examiner-algorithm fusion. The benefit was demonstrated under that study’s conditions and does not by itself establish admissibility or accuracy for every operational use.
In England and Wales, facial image comparison evidence has been admitted under the framework of R v. Atkins and Atkins (2009), in which the Court of Appeal confirmed that facial comparison evidence from a suitably qualified expert was admissible, provided the examiner disclosed the basis of the comparison and the limitations of the method.
England and Wales guidance stresses the subjective basis of image-comparison opinions, competing propositions, appropriate limitations and independent review. It permits verbal expression of significance, but cautions against statistical probabilities without an appropriate statistical basis. Specific legal requirements and current guidance should be checked for the case.
In India, expert-opinion evidence is addressed by section 39 of the Bharatiya Sakshya Adhiniyam 2023, subject to its application to the proceedings. A facial-comparison report should explain qualifications, methods, source imagery and limitations; the relevance of expert opinion does not itself validate a method.
Admissibility and scientific validation are distinct. Courts assess the proposed evidence under the applicable law; a reliable comparison also needs an appropriate method, suitable images and a reasoned account of observations and limitations.
CCTV Evidence and the Image Quality Problem
The dominant source of facial evidence in criminal investigations is CCTV footage rather than enrolment-quality photography, and it is captured at variable resolution, variable illumination, variable angle, and often with significant compression artefacts. The gap between NIST benchmark performance and operational performance on real CCTV evidence is the central practical challenge in forensic facial comparison.
Face-image suitability depends jointly on resolution, focus, pose, illumination, occlusion, motion and compression. Inter-pupillary distance in pixels can describe image scale, but it is not a universal pass/fail rule for automated recognition or forensic comparison.
Off-axis views can hide features and alter apparent proportions. Meaningful comparison requires assessment of what is actually visible and whether the reference images provide comparable views; one yaw-angle threshold cannot establish suitability for every task.
Before comparing images, document their source, processing and limitations, and assess whether the features needed for the task are visible. Expertise in image processing and expertise in facial comparison are distinct competencies.
A laboratory should use a documented, validated suitability procedure and record why imagery is usable or inadequate for the proposed task. Quality categories should be defined by that procedure and linked to the information needed for meaningful comparison.
Compression and copying can reduce or alter visible detail. Preserve the native recording where possible, record transformations and inspect the actual face imagery; camera specifications or codec names alone do not establish forensic suitability.
- Acquire and preserve the CCTV sourceObtain a forensic copy of the original digital video file from the recording device. Do not work from a re-recorded or compressed copy. Document the chain of custody, the device make and model, the compression codec and bitrate, and the date-time synchronisation status.
- Measure image qualityExtract suitable frames and document resolution, focus, pose, illumination, occlusion and compression. Use task-specific validated suitability criteria rather than universal pixel boundaries.
- Assess comparabilityAssess whether viewpoints, camera geometry and expressions permit meaningful feature comparison. Record processing and avoid assuming that alignment recovers obscured or missing information.
- Apply validated comparison procedureUse a validated morphological comparison procedure appropriate to the available features. Do not select photo-anthropometry or overlays as identification methods merely because a pose threshold is met.
- Record each feature findingRecord visible feature similarities, differences and limitations with standardized terminology. Feature counts alone do not determine evidential weight.
- Formulate and report conclusionReport the observations, competing propositions where applicable, uncertainty and limitations under the applicable validated reporting procedure. Do not present an unsupported numerical likelihood ratio.
- NIST FRTE / FATE (formerly FRVT)
- NIST’s face technology evaluation programmes: FRTE evaluates recognition and FATE evaluates analysis tasks. Historical reports use the name Face Recognition Vendor Test (FRVT).
- False positive identification rate (FPIR)
- At a stated threshold, the proportion of non-mated 1:N searches returning one or more candidates above threshold. It differs from pairwise false match rate and rank-k retrieval.
- Deep convolutional neural network (DCNN) embedding
- A numerical image representation produced by a deep convolutional neural network and compared using a similarity measure. Modern face-recognition architectures and representations vary.
- NGI face component
- The Interstate Photo System within NGI supports face-search leads. It is distinct from external repositories that FBI FACE Services may search under agreements.
- AFRS (India)
- NCRB’s proposed Automated Facial Recognition System, approved for implementation in 2020. Approval and procurement records do not by themselves establish nationwide commissioning.
- ENFSI BPM
- ENFSI’s 2018 Best Practice Manual for Facial Image Comparison recommends morphological analysis, validated procedures, attention to image limitations and transparent reporting.
- Morphological analysis
- The systematic comparison of qualitative facial feature shapes from a standardised feature list; the primary method in ENFSI BPM facial image comparison, applicable across a wider range of image quality conditions than photo-anthropometry.
- Photo-anthropometry
- Measurement of facial image landmarks and ratios. Perspective, pose, expression and landmark placement affect results; ENFSI does not recommend its use for comparison of uncontrolled imagery.
- IPD pixels
- Distance between pupil centres measured in image pixels, describing image scale. It does not by itself establish suitability or a universal recognition threshold.
- Verbal likelihood ratio (VLR)
- Verbal expressions can describe the strength of support for competing propositions under a defined reporting procedure. They should not be treated as a measured numerical likelihood ratio without calibration or as a universal ENFSI scale.
How should NISTIR 8280 (2019) demographic error findings be interpreted?
Can face recognition tell identical twins apart?
Does wearing a face mask defeat face recognition?
Sources
- 1.Best Practice Manual for Facial Image Comparison, 2018 · ENFSIMorphological comparison and method limitations
- 2.Face Technology Evaluations · NISTFRTE and FATE scope
- 3.Law Enforcement’s Use of Facial Recognition Technology · FBINGI-IPS and FACE Services repositories
- 4.Facial Comparison Overview and Methodology Guidelines, 2.1 · FISWGMorphological comparison and limits on overlays
Test yourself on Fingerprint Sciences with free, timed mocks.
Practice Fingerprint Sciences questionsFound this useful? Pass it along.