Video Frame Analysis and Enhancement

Analogue television standards (PAL at 25 fps, NTSC at 29.97 fps) use an interlaced scan pattern. The camera captures odd-numbered scan lines in one field and even-numbered lines in the next, each field at half the vertical resolution but at twice the frame rate. For a stationary subject, weaving the two fields together produces a clean full-resolution frame. For a moving subject, the two fields capture different moments and the result is the characteristic 'comb' artefact: horizontal stripes at moving edges.

Forensic de-interlacing must choose an appropriate algorithm. The three main approaches are: line doubling (each field is upscaled by duplicating each line, discarding the other field), field weave / blend (the two fields are combined with averaging, appropriate for static scenes), and motion-adaptive reconstruction (pixels in motion regions are taken from a single field; static regions use weave). Motion-adaptive is the most accurate for mixed scenes but requires a motion map, which is itself an estimation step.

A practical note: modern IP cameras record progressive images. De-interlacing is relevant for footage from analogue CCTV systems, older DVRs, and some hybrid systems that convert an analogue input. If the examiner receives an MP4 from a modern NVR, they should first verify whether the source camera was interlaced before applying de-interlacing; applying it to a progressive source can introduce artefacts.

Surveillance cameras in low-light conditions produce noisy images. Each pixel's value is the sum of the true scene luminance and a noise component that varies randomly from frame to frame. If the scene is static (or if motion is compensated), averaging N frames reduces the noise standard deviation by a factor of the square root of N. Average 16 frames and the noise is four times smaller; the signal-to-noise ratio improves by 12 dB.

Motion compensation is the key step for non-static scenes. If a subject moves between frames, naive averaging blurs them into a ghost image. The solution is to register the frames to a reference using an estimated motion field before averaging. For a camera that is moving slightly (camera shake), global registration (translation and rotation) may be sufficient. For a scene where the background is fixed but a subject moves, the examiner averages background regions across frames (improving the background) and handles the subject separately.

A camera samples the scene onto a pixel grid. Any detail finer than one pixel is lost, but it is not lost uniformly across all frames: if the camera (or the scene) moves slightly between frames, each frame samples the underlying scene at a slightly different sub-pixel position. Super-resolution algorithms estimate this sub-pixel motion and use it to reconstruct a single high-resolution image that has more samples per unit area of the scene than any single input frame.

The prerequisites are strict. Sub-pixel motion must actually be present (a rigidly mounted camera on a tripod with a completely static scene provides nothing for SR to work with). The motion must be accurately estimated. The number of independent frames must be sufficient (a 2x upscale requires approximately four independent sub-pixel shifts). These conditions are met more often than you might expect in real surveillance footage: even a fixed camera on a wall mount vibrates subtly, and that vibration is enough for SR to extract additional detail.

Validated forensic SR implementations include Amped FIVE's 'Super Resolution' module and Cognitech VideoActive. Both require the examiner to document the number of input frames, the motion estimation method, and the upscale factor. The output image is an estimate, not a measurement, and the report must say so.

Surveillance cameras often operate under mixed artificial lighting: sodium-vapour street lamps, LED shop lights, fluorescent tube ceiling lights, and incandescent spotlights can all illuminate the same scene. Each light source has a different spectral distribution and pushes the camera's automatic white-balance algorithm in a different direction. The result is that a blue jacket may appear black and a white shirt may appear yellow-green. Colour calibration corrects this by mapping the camera's output to a known colour standard.

The standard method is to photograph a colour reference chart (such as a ColorChecker Classic, which has 24 colour patches of known reflectance values) under the same lighting conditions as the original footage. The examiner computes a colour correction matrix that transforms the camera's response to the chart patches to the known values. This matrix is then applied to the footage. The result is a colour-corrected image that more accurately represents the true colours of the scene.

In casework, full colour calibration with a reference chart is only possible when the camera and lighting are still available and can be reproduced. When they are not, the examiner may be limited to scene-referred white-balance correction (using a known grey surface visible in the footage as a reference) or histogram-based equalisation. Both are less precise and must be described as approximate corrections in the report.

In a large installation, cameras may feed the same NVR and be hardware-synchronised, meaning all cameras capture frames at exactly the same moment. In practice, many investigations involve cameras from different systems: a shop camera, a street CCTV camera managed by a municipal authority, and a bank ATM camera, each with its own recorder and its own clock. Synchronising these requires finding common reference events.

Reference events can be: a person or vehicle visible simultaneously in the fields of view of two cameras; a flash of lightning or a police siren that produces a visible pulse in all recordings; an audio transient (a car horn, a shout) if the cameras have audio tracks; or a known-time event such as a digital display visible to one camera that can be matched to an event in another. For each reference event, the examiner records the timestamp in each camera's timeline and computes the offset. If multiple reference events are available, any drift between the cameras can be estimated.

Motion trajectory analysis reconstructs the path of a person or vehicle through a scene. In single-camera analysis this means annotating the subject's position (typically the feet, which are closer to the ground plane) in each frame or at regular intervals, producing an (x,y) track in the image coordinate system. If the camera's position, orientation, and lens parameters are known, this track can be projected onto a real-world coordinate system (a map or floor plan), yielding actual distances and speeds.

Camera calibration for ground-plane projection requires knowing the intrinsic parameters (focal length, principal point, lens distortion) and the extrinsic parameters (position and orientation of the camera in the world). These can be estimated from known reference distances in the scene (door widths, floor tile sizes, vehicle lengths) using photogrammetric methods. The result is an approximate projection; the examiner must report the estimated error in the ground-plane coordinates.

Object tracking across multiple cameras adds a step: trajectories from each camera are projected onto the same ground-plane map and the examiner checks whether they form a consistent, continuous path. Inconsistencies can reveal occlusions, errors in the synchronisation, or the presence of a second person whose path was confused with the primary subject's.

Transparency in forensic video enhancement means: preserving the original and all intermediate outputs; documenting every processing step with the software name, version, and parameters; and being able to reproduce the result from the original using the documented steps. These are not bureaucratic requirements. They exist because enhancement is a chain of decisions and each decision can affect the interpretation.

1. Preserve the original: never work on the original file. All processing is done on a working copy; the original and its hash are stored separately.
2. Log every step: record the tool name, version, function used, and all adjustable parameters for each processing operation.
3. Retain intermediate outputs: each stage's output is saved and hashed so the chain from original to final can be reproduced or audited.
4. State the purpose of each step: explain in plain language why each enhancement was applied and what it was intended to clarify.
5. State the limits: report what the enhancement cannot resolve. An enhanced face at 40 pixels wide may be clearer, but it may still be insufficient for positive identification.

Amped FIVE generates an automatic processing log that lists every filter applied, with parameters and timestamps. This log is admissible as part of the forensic report. Where other tools are used (ffmpeg, ImageJ, MATLAB scripts), the examiner writes the log manually. The key principle, drawn from OSAC (Organization of Scientific Area Committees for Forensic Science) guidance on video analysis, is that any other competent examiner following the documented steps should arrive at the same result.