Audio Enhancement and Speech Intelligibility

The most consequential decision in audio enhancement is made before any processing begins: classifying the noise. A spectral subtraction algorithm fed a stationary noise estimate removes tape hiss reliably and distorts a passing siren. The wrong method on the wrong noise type introduces processing artefacts that reduce intelligibility rather than improving it.

In practice, recordings often contain layered noise: a fixed HVAC background with intermittent vehicle pass-bys. The examiner processes in stages, removing the stationary component first, then addressing the residual non-stationary interference. Each stage is saved as a separate intermediate file so the chain of processing is auditable.

Spectral subtraction, introduced by Boll in 1979, works in the frequency domain. For each short frame of audio, the magnitude spectrum of the noise estimate is subtracted from the magnitude spectrum of the noisy frame. The phase of the original frame is retained, and the result is inverse-transformed back to a time-domain signal. The noise estimate is typically derived from a few seconds of recording where no speech is present (silence before a conversation begins, a pause between sentences).

The main artefact is musical noise: residual tones scattered across the spectrum that sound like random beeping or a distant choir. They arise when the subtraction overshoots (subtracting more noise energy than was actually present in a particular frame) or when the noise estimate does not match the instantaneous noise level. Over-subtraction factors and spectral flooring (not letting the subtracted spectrum go below a small fraction of the noise estimate) reduce musical noise at the cost of leaving some residual background.

The Wiener filter takes a more principled approach. It computes, in each frequency bin, the gain that minimises the mean-squared error between the estimated clean signal and the noisy observation. The gain is driven by the local signal-to-noise ratio: in bins where speech dominates, the gain is close to 1 (pass most of the signal); in bins where noise dominates, the gain is close to 0 (attenuate heavily). The result is perceptually smoother but still depends critically on the quality of the noise power spectral density estimate.

Adaptive filtering addresses interference that varies over time. Rather than deriving a fixed noise estimate from a single silent interval, the filter updates its coefficients continuously as the signal progresses, tracking changing noise characteristics. The most widely used adaptive algorithm in audio forensics is the least-mean-squares (LMS) filter and its variants, which adjust their tap weights after each new sample to reduce the error between the filter's output and a desired signal.

A reference signal is required: a microphone placed to capture mainly the interfering source with minimal speech capture. The adaptive filter models the path from that reference to the primary microphone and subtracts the estimated interference, the same principle used in noise-cancelling headsets. In forensic casework a dedicated reference microphone is almost never available, because the recording was not made for this purpose. Examiners must instead rely on time-frequency masking or multi-channel analysis if a second channel is available.

Traditional analogue telephony and many digital codecs (G.711, GSM-FR) limit audio bandwidth to approximately 300-3400 Hz, dropping the low-frequency energy below 300 Hz and the high-frequency consonant energy above 3400 Hz. The missing high-frequency band degrades fricative and affricate discrimination: /s/ and /f/, /ch/ and /sh/ become harder to distinguish. Bandwidth extension attempts to regenerate the missing frequency content from correlates present in the narrowband signal.

Classical bandwidth extension exploits the approximate periodicity of voiced speech harmonics: if a voiced segment has a fundamental frequency of 120 Hz, harmonics at 240 Hz, 360 Hz, and beyond are mathematically related to harmonics already present in the 300-3400 Hz band. Spectral folding and excitation estimation can reconstruct plausible high-frequency content. Machine-learning approaches train on narrowband/wideband recording pairs and learn to predict the missing spectral envelope.

Before objective measures existed, evaluating enhancement quality required listening panels, which are slow, costly, and variable. Two standardised instrumental metrics have largely replaced listening panels for benchmarking:

• PESQ (ITU-T P.862): Perceptual Evaluation of Speech Quality, originally designed to assess telephone network degradation. It compares a degraded signal against a clean reference and produces a Mean Opinion Score equivalent on a 1-4.5 scale. PESQ requires a clean reference recording of the same utterance, which is rarely available in casework, so its forensic application is mainly for validating processing chains in controlled tests.
• STOI (Short-Time Objective Intelligibility): computes the correlation between short-time temporal envelopes of the clean and processed signals in one-third-octave bands. Scores range from 0 to 1 and correlate well with the proportion of words correctly identified by listeners. STOI also requires a clean reference but is more sensitive than PESQ to intelligibility changes rather than quality changes.

In practical forensic work, reference signals are rarely available. Examiners instead use informal A-B listening comparisons, documented transcription trials by trained listeners, and STOI estimates computed using noise-estimation-based surrogates for the reference. The goal is a documented, reproducible assessment of whether the enhancement improved the ability to understand speech.

The SWGDE Best Practices for Forensic Audio (most recent version 2.5, published 2022) and equivalent guidance from the Audio Engineering Society (AES SC-03-12) require that the following be documented and disclosed for any enhanced recording submitted as evidence:

1. Original preservation: a bit-for-bit copy of the original recording must be made and verified against its hash before any processing begins. All subsequent work is on working copies.
2. Processing log: every processing step is documented in order: the tool used, version, parameters, and the purpose of each step. The log must be complete enough that another qualified examiner could reproduce the result.
3. Working-copy labelling: the enhanced file is clearly labelled as a working copy, not a substitute for the original. Both are submitted to court.
4. AGC caution: automatic gain control should be applied, if at all, only as a final step. Premature AGC amplifies background noise during speech gaps, which can obscure quiet words that were present.