Evidence Collection & Testing

Types of Audit Evidence

Understanding Audit Evidence

Audit evidence is the information collected by auditors to support their findings, conclusions, and recommendations. The quality and sufficiency of evidence directly determines the credibility of the audit report. Evidence must be relevant (it relates to the control being tested), reliable (it comes from a trustworthy source), and sufficient (there is enough of it to support the conclusion). Weak or insufficient evidence undermines audit findings and can lead to disputed results or regulatory rejection.

Categories of Audit Evidence

Audit evidence can be categorized by its source and nature. Understanding these categories helps auditors select the most appropriate and reliable evidence for each control being evaluated. In general, evidence obtained directly by the auditor is more reliable than evidence provided by the auditee, and documentary evidence is more reliable than verbal assertions.

• Corroboration: Use multiple types of evidence to support a single finding. For example, corroborate an interview response with documentary evidence and system logs.
• Independence: Evidence obtained independently by the auditor (such as direct observation or re-performance) is more reliable than evidence provided by the auditee.
• Timeliness: Evidence should be relevant to the audit period under review. A screenshot of current settings does not prove the control was operating throughout the entire audit period.
• Completeness: Evidence should cover the entire population or a statistically valid sample, not just a few hand-picked examples the auditee provides.
• Chain of Custody: Maintain a clear record of when evidence was collected, from whom, and how it has been stored to preserve its integrity.

Sampling Methods & Control Testing

Why Sampling Is Necessary

In most audits, it is impractical to test every single instance of a control operating over the entire audit period. If a company processes 10,000 access requests per year, the auditor cannot review all 10,000 individually. Sampling allows auditors to draw conclusions about an entire population by examining a representative subset. The key is that the sample must be selected in a way that is representative, unbiased, and large enough to provide reasonable assurance about the control's effectiveness across the full population.

Common Sampling Methods

Determining Sample Size

Sample size depends on several factors including the size of the population, the desired level of assurance, the expected deviation rate, and the tolerable deviation rate. Professional standards such as ISACA's IS Audit and Assurance Standards and the AICPA's audit sampling guidelines provide frameworks for determining appropriate sample sizes. As a general guideline for IT audits, many firms use the following approach based on population size and control frequency.

Control Testing Approaches

• Inquiry: Ask the control owner how the control operates. This is the weakest test and should always be corroborated with other methods.
• Observation: Watch the control being performed in real time. For example, observe an administrator performing a user access review.
• Inspection: Examine documentary evidence that the control was performed. For example, review signed access request forms, change approval records, or patch deployment logs.
• Re-performance: The auditor independently performs the control procedure to verify it produces the expected result. For example, attempt to log in with a disabled account to verify it is truly disabled.
• Computer-Assisted Audit Techniques (CAATs): Use automated tools to test controls across the entire population rather than a sample. For example, run a script to identify all accounts without MFA enabled.

Automated Tools, Interviews & Working Papers

Leveraging Automated Audit Tools

Automated tools significantly improve the efficiency, consistency, and coverage of cybersecurity audits. They can analyze configurations, scan for vulnerabilities, extract system data, and test controls across entire populations in minutes — work that would take days or weeks to perform manually. However, automated tools are a means of gathering evidence, not a substitute for professional judgment. The auditor must still interpret the results, assess their significance, and determine whether findings represent genuine control deficiencies or false positives.

• Vulnerability Scanners (Nessus, Qualys, Rapid7): Identify known vulnerabilities in systems, applications, and network devices. Provide evidence of the organization's patch management and hardening effectiveness.
• Configuration Assessment Tools (CIS-CAT, Microsoft Secure Score, AWS Security Hub): Evaluate system configurations against established benchmarks and best practices. Provide evidence of configuration management and hardening controls.
• Log Analysis and SIEM Tools (Splunk, ELK Stack, Microsoft Sentinel): Aggregate and analyze log data from multiple sources. Provide evidence of monitoring, detection, and incident response capabilities.
• Identity and Access Analysis Tools (Scripts, IAM Reports, Access Certification Platforms): Extract and analyze user access data to identify excessive privileges, orphaned accounts, and separation of duties violations.
• Network Analysis Tools (Wireshark, Nmap): Map network topology, identify open ports and services, and verify network segmentation. Provide evidence of network security controls.

Conducting Effective Interviews

Interviews are a primary evidence-gathering technique, particularly for understanding processes, identifying gaps, and validating documentary evidence. While interview responses alone are the weakest form of evidence, a well-conducted interview can reveal control weaknesses that are not apparent from documentation review alone. Effective interviewing requires preparation, active listening, and the ability to ask probing follow-up questions without making the interviewee defensive.

• Preparation: Review relevant policies, procedures, and prior audit findings before the interview. Prepare a list of topics and questions but allow flexibility for follow-up questions based on responses.
• Open-Ended Questions: Start with open-ended questions such as 'Walk me through how you process an access request' rather than leading questions that suggest the expected answer.
• Corroboration Requests: When an interviewee describes a process, ask for evidence: 'Can you show me an example of a recent access request that went through this process?'
• Documentation: Take detailed notes during the interview and document who was present, the date, and the topics discussed. Consider asking permission to record the interview for accuracy.
• Multiple Perspectives: Interview people at different levels — managers describe the intended process, while the staff who execute it daily describe the actual process. Discrepancies between these are important findings.

Organizing Working Papers

Working papers are the complete record of the audit — they document the planning, evidence collected, testing performed, findings identified, and conclusions reached. Well-organized working papers allow another qualified auditor to understand the work performed and reach the same conclusions without needing to redo the audit. They also provide a defensible record if audit findings are challenged by the client or reviewed by regulators.

• Audit Program: The detailed plan listing every control to be tested, the testing procedure, the expected evidence, and the assigned auditor.
• Evidence Repository: An organized collection of all evidence gathered, including documents, screenshots, log extracts, scan reports, and interview notes. Each piece of evidence should be labeled and cross-referenced to the control it supports.
• Testing Worksheets: For each control tested, document the test objective, the sample selected, the testing steps performed, the results observed, and the conclusion (effective, ineffective, or not applicable).
• Finding Documentation: For each identified finding, document the condition (what was found), the criteria (what was expected), the cause (why the gap exists), the effect (the risk or impact), and the recommendation.
• Review Notes: Supervisor review comments and the auditor's responses, demonstrating quality review of the work performed.