C_llr (Log-Likelihood Ratio Cost)
Definition
A proper scoring rule measuring the calibration quality of a speaker recognition system's LR outputs; a well-calibrated system has C_llr close to zero. Required as a disclosure metric under ENFSI BPM 2022 for systems used in forensic casework.
- Measures
- Calibration quality of LR outputs
- Type
- Proper scoring rule
- Ideal value
- Close to zero for well-calibrated systems
- Required by
- ENFSI BPM 2022 for forensic casework
Common questions
Why does a speaker recognition system need a calibration metric like C_llr at all?+
A system can rank same-speaker and different-speaker pairs correctly yet still report likelihood ratios that overstate or understate the real strength of the evidence, and C_llr specifically penalises that miscalibration rather than just discrimination ability, which matters because courts rely on the numeric LR value, not just a rank order.
How is C_llr different from discrimination metrics like the equal error rate?+
Equal error rate only measures how well the system separates same-speaker from different-speaker pairs at a threshold, while C_llr also penalises overconfident or underconfident likelihood ratios, so a system can have a good error rate but a poor C_llr if its output numbers are not properly calibrated.
Related terms
- Background Population Database
- A corpus of speaker recordings used to calibrate the LR in a forensic comparison. Must be matched to the case in language,...
- BATVOX
- A commercial automated speaker recognition system (originally Agnitio, later Nuance Communications) widely used in operational forensic casework in Europe and Latin America....
- ENFSI BPM
- The ENFSI Best Practice Manual for Forensic Comparison of Speech (2015, rev. 2022); the operational standard for speaker comparison in European forensic...
- ENFSI Verbal Scale
- A standardised table published by the European Network of Forensic Science Institutes that maps ranges of LR values to verbal expressions of...
- Equal Error Rate (EER)
- The point on a classifier's detection error tradeoff curve where the false accept rate equals the false reject rate. Lower EER indicates...
- I-Vector
- A fixed-length low-dimensional speaker representation extracted by projecting a speaker's UBM statistics into a total variability space. Introduced by Dehak et al....
- NIST SRE (Speaker Recognition Evaluation)
- A competitive benchmarking series run by the National Institute of Standards and Technology since 1996, in which research and commercial teams submit...
- PLDA (Probabilistic Linear Discriminant Analysis)
- A statistical back-end model used with i-vector and x-vector systems to compute a similarity score between two utterance representations, normalised for within-speaker...
- X-Vector
- A deep-learning speaker embedding produced by the penultimate layer of a time-delay neural network (TDNN) trained for speaker classification. Introduced by Snyder...