Fieldwork, Evidence Collection, and Control Testing

Security audit fieldwork uses four techniques, each suited to gathering different types of evidence. Skilled auditors use them in combination, triangulating findings across multiple sources rather than relying on any single type of evidence.

Interviews are typically the first fieldwork technique applied. The auditor meets with control owners, system administrators, and relevant managers to understand how each control is supposed to work, who is responsible for it, how exceptions are handled, and whether staff are aware of the relevant policy. Interview notes are evidence, but they are low-reliability evidence on their own because they depend on the interviewee's accuracy and honesty. The auditor uses interview findings to direct subsequent testing rather than to support findings directly.

Configuration inspection is the most technically demanding technique. The auditor examines system settings directly: firewall rule sets, access control lists, password policy parameters, encryption settings, patch levels, and audit log configurations. This requires either direct system access (with appropriate controls on what the auditor can do) or the use of configuration management tools that export current state. The output is objective evidence of the control's current state, but it is a point-in-time snapshot. A misconfigured system corrected minutes before the auditor's inspection will not show up as a finding.

Audit standards require evidence to satisfy three criteria before it can support a finding. ISO 19011 frames these as relevance, reliability, and sufficiency. ISACA's audit standards use similar language. Understanding what each criterion means in practice is essential for deciding when to stop collecting evidence and when to keep going.

Relevance asks whether the evidence relates directly to the control being tested. A firewall log showing blocked traffic from an external IP is relevant to a finding about perimeter access control. The same log is not relevant to a finding about patch management. Auditors sometimes collect evidence opportunistically, capturing anything that looks interesting, and end up with large volumes of material that does not support any specific finding. Relevance keeps the evidence collection focused.

Reliability relates to the source and how the evidence was gathered. Evidence obtained directly by the auditor through system inspection or observation is more reliable than evidence provided by the auditee. Among documents provided by the auditee, those generated automatically by systems (access logs, change records) are more reliable than manually prepared summaries. External confirmations, such as a certificate from a third-party penetration testing firm or a vendor security attestation, sit somewhere between the two. The reliability of evidence affects how much of it is needed to reach a sufficient conclusion.

Sufficiency is a judgment call, not a formula. The auditor must decide whether the evidence collected, in its totality, supports the conclusion with reasonable assurance. For a frequently executed control such as a daily backup verification, a sample of 25 daily logs over a quarter is generally considered sufficient if no exceptions appear. For a quarterly management review, the auditor typically examines all occurrences over the audit period rather than sampling. Where the risk is high or prior audits have found exceptions, a larger sample is warranted.

Forensic investigations apply formal chain of custody procedures to ensure that evidence is admissible in legal proceedings and has not been tampered with. The same logic applies to audit evidence, though the formality is lower and the consequences of a chain of custody failure are professional and reputational rather than criminal. An auditor who cannot demonstrate that a screenshot was taken on a specific date from a specific system, or that a configuration extract has not been modified since collection, has a finding that management can reasonably dispute.

Audit chain of custody documentation should record: the date and time the evidence was collected, the name of the auditor who collected it, the system or source it came from, the method used to collect it (direct observation, screen capture, system export, document request), and any transformation applied to it (for example, a configuration file converted from a proprietary format to plain text for review). Most professional audit management platforms, such as TeamMate, AuditBoard, or Workiva, create automatic timestamps and access logs that satisfy this requirement.

When evidence is provided by the auditee rather than collected directly by the auditor, the documentation should also record who provided it and when. This matters if the auditee later claims the document was not authorised for release, was a draft, or has since been superseded. Screenshots should capture the system date and time in the image itself where possible. Configuration exports should be hash-verified if the audit is likely to result in disputed findings.

In jurisdictions where audit findings may be reviewed by a regulator or used in enforcement proceedings, the standard for evidence documentation is higher. The GDPR supervisory authority process in the European Union, the Securities and Exchange Board of India (SEBI) inspection framework, the Financial Conduct Authority (FCA) regulatory visit process in the United Kingdom, and the SEC examination process in the United States all have specific expectations about what documentation an audited entity must be able to produce. An organisation whose internal audit function maintains disciplined chain of custody is better prepared for regulatory visits than one that does not.

A control's design is effective if, operating as described, it would prevent or detect the risk it targets. Testing design effectiveness is primarily a documentation and walkthrough exercise. The auditor reads the policy or procedure, traces its logic, and asks whether the described steps, if followed correctly, would achieve the stated objective. For technical controls, the auditor reviews the intended configuration and asks whether those settings, in place, would enforce the required behaviour.

A walkthrough is the most common design effectiveness test. The auditor selects one representative transaction or event, such as a single access request, a single change management ticket, or a single patch deployment, and traces it through every step of the stated process, confirming at each step that the required action was taken and documented. The walkthrough does not confirm that the control operates consistently across all transactions; it confirms only that the process works for one case and that the relevant people know what to do. It is a necessary but not sufficient basis for concluding the control is effective.

Design deficiencies are the more serious finding. A control that is designed wrong will fail regardless of how well staff follow it. Common design deficiencies include: an access review process that checks whether accounts exist but not whether the associated permissions are appropriate; a patch management process that covers servers but excludes network devices; an encryption requirement that applies to data at rest but not data in transit between internal systems. These gaps require a recommendation that the control itself be redesigned, not just better executed.

Operating effectiveness testing asks whether a control has consistently operated as designed throughout the audit period, not just at the moment of inspection. It requires evidence of the control executing across multiple instances, and it typically uses sampling because the full population is too large to examine completely. The size of the sample depends on the frequency of the control, the risk it addresses, and the confidence level required.

Sampling guidance used in information security audits typically draws on statistical or attribute sampling methods. ISACA's guidance recommends samples of 25 items for controls that operate daily, 15 items for weekly controls, 5 items for monthly controls, and 2 items for quarterly controls, when the auditor requires a confidence level of 90 to 95 percent and no exceptions are expected. If prior audits have found exceptions, the sample size should be increased. These are starting points, not mandatory thresholds; the auditor adjusts based on the specific control and context.

Re-performance is a high-reliability technique for operating effectiveness testing. Rather than accepting the auditee's records as evidence that a control operated, the auditor performs the control independently and compares the result. For example, the auditor independently reviews the access rights of a sample of user accounts and compares the result to the organisation's most recent access review records. If the auditor's independent check finds the same accounts the internal review found, the review process is operating effectively. If the auditor finds active accounts that the internal review cleared but which should have been removed, the control has an operating failure.

The distinction between design and operating effectiveness matters for how a finding is characterised and what remediation is recommended. A control with a design deficiency needs to be rebuilt. A control with an operating effectiveness failure may need stronger supervision, automation, or training rather than a redesign. Some controls fail on both dimensions, which is the most serious finding category.

A control cannot be rated as deficient without a stated criterion. The criterion tells the auditor what the control is supposed to achieve, and the finding explains how the observed state fails to meet it. Criteria come from several sources. External standards such as ISO 27001 Annex A define specific controls the organisation has committed to implement. The NIST Cybersecurity Framework defines outcomes that effective controls should achieve. Regulations such as GDPR Article 32, HIPAA 45 CFR Part 164, and PCI-DSS Requirement 6.3 state specific technical and process requirements. The organisation's own policies and procedures are always criteria, because management has represented to its board and regulators that those policies are followed.

Auditors typically rate each tested control using a simple classification. A control is effective if it meets the criterion consistently across the sampling period with no unexplained exceptions. It is partially effective if it meets the criterion most of the time but has documented exceptions that are not explained by authorised deviations. It is ineffective if the criterion is not met, if the control is not operating at all, or if the design would not achieve the objective even if followed correctly.

Each finding in the audit report must state the criterion, the condition observed, the cause of the gap (where it can be determined), and the potential effect. This structure, sometimes called the finding elements or CCEA (criterion, condition, effect, cause), ensures that management understands not just what is wrong but why it matters and what they need to fix. A finding that says only 'patch management is inadequate' without a criterion, condition, and effect is not actionable and will not drive remediation.