Deepfake Detection: Signal, Physiological, and Semantic Methods

When a GAN generator upsamples its internal feature maps to produce a full-resolution image, it typically uses transposed convolution (also called deconvolution). This operation distributes values from a smaller grid onto a larger one by inserting zeros between elements and then applying a convolution.

Because the filter kernels cover these inserted zeros unevenly, the output has a periodic pattern of varying energy at a spatial frequency determined by the upsampling stride. In an image this manifests as a faint regular grid, invisible to the eye but clearly present in a 2-D Fourier transform as a set of peaks at predictable locations.

Frank et al. (2020) demonstrated that a simple classifier trained on the DCT (Discrete Cosine Transform) spectrum of face crops achieves near-perfect accuracy on multiple GAN architectures, with frequency-domain classifiers outperforming classifiers trained in the pixel domain for cross-architecture generalisation. The limitation is that diffusion models and flow-based generators have different upsampling operations and do not produce the same checkerboard pattern. Frequency-domain detectors trained on GAN artefacts perform poorly on diffusion model output, reinforcing the generalisation problem.

Human bodies have involuntary, temporally structured behaviour that is hard to synthesise. Early GAN face generators were trained predominantly on open-eye portraits, which meant that generated videos had abnormally low eye-blink rates and unnatural blink dynamics.

Li et al. (2018) exploited this, building a blink-detection module that classified videos based on whether the blink frequency and blink shape matched statistical priors for real subjects. This was one of the first published detectors and was effective against the generation methods available at that time.

• Eye-blink rate: early GANs under-produced blinking; subsequent generators added explicit blink modelling, narrowing this gap but not eliminating it.
• rPPG signal: authentic faces show a periodic subtle colour oscillation in the skin, at roughly the heart rate, detectable by averaging pixel values in cheek regions over time. Synthesised faces typically show no such coherent oscillation or show it only spuriously.
• Head micro-motion: the cardiac cycle produces small, periodic head motion visible in video. Face-swap pipelines applied to real video will carry genuine head motion, but fully synthesised talking heads often have motion vector statistics inconsistent with this physiological source.
• Gaze and pupil response: pupil dilation responses to scene lighting and gaze direction consistency with scene geometry are difficult to synthesise and provide useful secondary signals in high-resolution video.

Facial landmark detection provides a fast, interpretable way to flag geometric irregularities. A set of 68 or 478 landmarks covering the eyes, nose, mouth, and jaw outline can be fitted to any face image.

For authentic faces, the inter-landmark distances and angles fall within tight statistical bounds established from large datasets of real subjects. Synthesised faces, particularly those produced by older or lower-quality generation pipelines, show landmark configurations that deviate from these bounds, especially at the outer corners of the eyes, the ear region, and the nose-to-lip distance.

Beyond individual frames, temporal landmark trajectories in video carry information. A real talking head has facial muscle activations that follow biomechanical constraints: the orbicularis oculi contracts when smiling, the jaw moves in a predictable arc during speech. Synthesised videos often violate these soft constraints, producing landmark trajectories that are statistically improbable for genuine face motion.

The SRM (Steganalysis Rich Model) approach, originally developed for image steganography detection, was applied to deepfake detection by several research groups from 2019 onward. The key idea is to compute a high-pass residual image, the original minus a smoothed version, which removes low-frequency content and amplifies the subtle statistical texture left by processing operations. A CNN trained on residual images can learn to distinguish camera-native noise from generation-pipeline noise.

Cozzolino and Verdoliva's Noiseprint (2020) extends this idea by framing it as camera-model attribution. A CNN is trained to extract a fingerprint that identifies the camera model used to take a photograph, analogous to PRNU (photo-response non-uniformity) in traditional digital camera forensics.

When applied to a deepfake image, the face region shows an absent or anomalous fingerprint, because it was synthesised rather than captured. The background shows a consistent camera fingerprint if it came from genuine source video. The boundary between these two regions is the detection signal.

Semantic inconsistencies are failures of plausibility at the content level. Unlike spectral or residual methods, they do not require a trained classifier or a Fourier transform; they require knowledge of what faces, bodies, and scenes look like. Teeth are a canonical example: generating a realistic arrangement with correct occlusion, spacing, and lighting response is a consistently hard problem for generative models, and high-resolution deepfakes often show smeared or incorrectly shaped teeth in wide-open-mouth frames.

• Illumination geometry: the direction of specular highlights on the skin and the direction of cast shadows should be geometrically consistent with each other and with the background scene. Inconsistent lighting direction is a reliable manual review signal.
• Reflection in eyes: the corneal specular reflection (the small bright spot on the eye surface) should match the room's light sources and their positions. Mismatched or absent corneal reflections indicate the face was synthesised or composited from a different lighting environment.
• Jewellery and accessories: rings, earrings, and glasses frames are structurally regular objects. Generators often warp or partially dissolve them, particularly near the skin-to-object boundary.
• Temporal semantic consistency: an earring that changes shape between frames, a necklace that flickers, or a scar that appears and disappears are temporal semantic anomalies invisible in any single frame.

The detection generalisation problem is well-documented in benchmark studies. Rossler et al. (2019) introduced FaceForensics++, a dataset with videos produced by four manipulation methods. Detectors trained on one manipulation method dropped in accuracy substantially when tested on a different one. The Cross-Generator Generalisation test, used in the FaceForensics++ and DFDC (DeepFake Detection Challenge) benchmarks, shows that accuracy on unseen generator families is substantially lower than accuracy on held-out examples of the same generator.

Proposed approaches to improve generalisation include frequency-domain training, which learns generation-family-agnostic spectral features rather than generator-specific pixel patterns; self-supervised contrastive pre-training on real-world image augmentations; and ensemble detectors that aggregate signals from multiple feature families. None of these fully resolves the problem, particularly against adversarially post-processed deepfakes where JPEG compression or film-grain overlays deliberately suppress spectral artefacts.

In practice, forensic conclusions based on deepfake detection should state the specific detector or method applied, the generator families it was validated against, the test conditions, and the confidence bounds on the result. Presenting a detector output without these qualifications overstates what the technology can actually deliver. The same qualifications carry into court, where testimony framed as bounded consistency opinions with documented error rates meets admissibility standards that a categorical claim of synthesis does not.