Network Security & IPSec

Security mechanisms are the defences that generate evidence — firewall logs, IDS alerts, VPN tunnels. This chapter covers the core security architecture from IPSec to SSL/TLS, so you understand both how traffic is protected and what records each mechanism keeps.

9.1IPSec

IPSec (Internet Protocol Security) is a suite of protocols that encrypts and authenticates IP packets at Layer 3 of the OSI model. Because it operates at the network layer, it protects all traffic — TCP, UDP, ICMP — without requiring individual applications to be aware of it.

IPSec runs in two modes depending on who is communicating:

Transport mode encrypts only the IP payload — the TCP/UDP content inside the packet. The original IP header remains in cleartext, so routers along the path can still read the source and destination addresses. Transport mode is used for host-to-host communication where both endpoints support IPSec directly.

Tunnel mode encrypts the entire original IP packet and wraps it inside a new IP packet with a new outer header. The original source and destination are hidden from anyone examining the outer packet. Tunnel mode is used for gateway-to-gateway VPN connections, where two firewalls at the edges of two networks build an encrypted tunnel and all traffic between the networks travels through it.

IPSec provides its security through two sub-protocols:

• AH (Authentication Header) — protocol number 51. Provides authentication and integrity: it guarantees the packet came from the claimed sender and was not modified in transit. AH does not provide confidentiality — the payload is not encrypted. Because AH authenticates part of the outer IP header, it is incompatible with NAT (NAT changes the IP header and breaks the hash).
• ESP (Encapsulating Security Payload) — protocol number 50. Provides encryption, authentication, and integrity. ESP is the protocol used in virtually all real-world VPN deployments because it actually hides the payload content.

IKE (Internet Key Exchange) runs on UDP port 500 and handles the negotiation phase before any encrypted traffic flows. IKE establishes a Security Association (SA) — a set of agreed parameters (algorithm, key, lifetime) — between the two peers and securely exchanges the session keys. IKEv2 is the current version.

Forensic relevance: IPSec tunnel traffic appears as a stream of encrypted ESP packets in a packet capture — the payload is opaque noise. To reconstruct what was communicated, the examiner needs the SA keys (rarely available) or must rely on IKE daemon logs, which record tunnel establishment time, peer IP addresses, and negotiated parameters. Endpoint OS logs record when tunnels were brought up and torn down.

9.2Firewalls & DMZ

A firewall enforces an access control policy on network traffic. Every packet or connection is compared against an ordered rule set — if a matching rule says "allow," the traffic passes; if "deny," it is dropped. A default-deny policy drops anything not explicitly allowed.

Types of firewall by inspection depth:

Packet filter (stateless) — inspects each packet in isolation, checking source IP, destination IP, port, and protocol. It is fast and simple but has no memory of prior packets. Attackers can sometimes fool it with fragmented packets or spoofed source IPs.

Stateful inspection — tracks connection state (SYN → ESTABLISHED → FIN). It knows whether an incoming packet belongs to a legitimate established connection. Spoofing a stateful firewall is significantly harder.

Application-layer / proxy firewall (WAF) — understands specific protocols (HTTP, FTP, DNS) at Layer 7. It can inspect content and detect SQL injection and XSS attacks embedded in HTTP requests.

NGFW (Next-Generation Firewall) — combines stateful inspection with deep packet inspection (DPI), an inline IPS, SSL/TLS decryption and inspection, application identification (can block Tor or BitTorrent regardless of port), and user-identity awareness.

DMZ (Demilitarised Zone) is a network segment positioned between the untrusted internet and the trusted internal LAN, protected by two firewalls — one facing the internet and one facing the internal network. Servers that must be reachable from the internet (web servers, mail servers, DNS) sit in the DMZ. If an attacker compromises a DMZ server they are still blocked from the internal LAN by the internal firewall.

Firewall logs record every allowed and denied connection: timestamp, source IP, destination IP, source port, destination port, protocol, bytes transferred, and rule name. These logs are the primary evidence source in network intrusion investigations — they show which IP first contacted a compromised server, which internal hosts phoned home to a malicious IP, and the volume of data transferred.

9.3IDS & IDPS

An IDS (Intrusion Detection System) passively monitors network traffic or host activity and generates alerts when it detects suspicious patterns. It does not block traffic — it only observes and reports. An analyst reviews the alerts and decides what action to take.

An IPS / IDPS (Intrusion Prevention System) sits inline in the traffic path so it can actively drop or modify malicious packets in real time. Because it is inline, a high false-positive rate or misconfiguration can disrupt legitimate traffic — tuning is critical.

Detection methods:

Signature-based detection works like antivirus software: each attack has a known pattern (signature), and the IDS compares traffic against a library of signatures. It is fast and produces very few false positives on known attacks. Its weakness is that it cannot detect zero-day attacks that have no signature yet.

Anomaly-based (statistical) detection builds a baseline of what "normal" traffic looks like — normal volume, port distributions, protocol ratios. When traffic deviates significantly from the baseline an alert fires. This can detect novel attacks but produces more false positives, because legitimate traffic sometimes deviates from the baseline (e.g., a software update push or a video call).

Placement:

• NIDS (Network-based IDS) connects to a SPAN/mirror port on a switch and receives a copy of all traffic without interrupting the flow. It sees all conversations on that segment.
• HIDS (Host-based IDS) runs as software on an individual endpoint. It monitors system calls, file integrity (comparing current file hashes against known-good values), and log entries. HIDS can detect threats in encrypted traffic that a NIDS would see only as ciphertext.

SNORT is an open-source, rule-based NIDS widely used in India and tested in FACT examinations. A Snort rule specifies: protocol, source/destination IP range, source/destination port, direction of traffic, and content patterns to match. Rules are distributed as rule sets (e.g., emerging-threats, community rules) updated daily.

Evidence value: IDS alert logs include timestamp, alert name, severity, source IP, destination IP, ports, protocol, and a raw excerpt of the matching packet. In an intrusion investigation these logs establish the earliest known moment of attack activity and identify the attacker's IP — typically the first evidence the investigator examines.

9.4Authentication Protocols

Kerberos is the ticket-based authentication protocol used in Windows Active Directory environments. It involves three parties: the client, the Key Distribution Centre (KDC) (a domain controller), and the target service. The client's password never travels over the network; instead:

1. The client requests a Ticket-Granting Ticket (TGT) from the KDC by proving knowledge of its password hash.
2. With the TGT, the client requests a service ticket for a specific resource (e.g., a file share or database server).
3. The client presents the service ticket to the target service, which validates it directly.

Tickets expire by default after 10 hours, limiting the window for replay attacks. Key Windows event IDs: 4768 = TGT requested, 4769 = service ticket requested, 4771 = Kerberos pre-authentication failed (wrong password). The Kerberoasting attack requests service tickets for accounts that have a Service Principal Name (SPN) registered, then cracks those tickets offline. Detection indicator: Event ID 4769 with encryption type 0x17 (RC4 downgrade).

X.509 Certificates define the standard format for public key certificates. An X.509 certificate binds a public key to an identity and is signed by a trusted Certificate Authority (CA). Key fields: subject name, issuer (CA) name, serial number, validity period (Not Before / Not After), public key, and CA digital signature. Used in TLS for server authentication. Forensically, the certificate serial number and issuing CA recorded in TLS handshake logs prove which server identity was presented during an encrypted session — even if the payload is opaque.

LDAP (Lightweight Directory Access Protocol) is the protocol for querying and modifying directory services such as Active Directory. Port 389 (cleartext) or 636 (LDAPS, TLS-encrypted). LDAP stores user accounts, group memberships, password hashes, and Group Policy Objects. Forensically, LDAP query logs reveal reconnaissance: an attacker who has obtained initial access often runs LDAP queries to enumerate all users, groups, and computers — a pattern detectable in directory service event logs.

Multi-Factor Authentication (MFA) requires at least two of: something you know (password), something you have (OTP token, hardware key, push notification), or something you are (biometric). Even if a password is phished, the attacker cannot authenticate without the second factor. Forensically, failed MFA attempts — repeated OTP failures or unexpected push notification responses — appear in authentication logs and indicate an account takeover attempt in progress.

9.5SSL/TLS, SET & VPN

SSL/TLS encrypts application-layer traffic so that data between a browser and a web server (or any two TLS-enabled applications) cannot be read or modified by a network eavesdropper. TLS 1.2 and TLS 1.3 are the current versions. SSL 3.0, TLS 1.0, and TLS 1.1 are deprecated and should never appear in modern logs — their presence is itself a forensic finding (outdated server or active protocol downgrade attack).

TLS Handshake — 5 steps:

1. Client Hello — the client sends the highest TLS version it supports, a list of cipher suites, and a random nonce (client random).
2. Server Hello + Certificate — the server selects a cipher suite, sends its own random nonce, and presents its X.509 certificate for the client to verify.
3. Client verifies certificate + Key exchange — the client checks the certificate against its trusted CA store. In TLS 1.2 it sends a pre-master secret encrypted with the server's public key; in TLS 1.3 it uses an ephemeral Diffie-Hellman key share.
4. Session keys derived — both sides independently derive the same symmetric session keys from the exchanged values. No key is ever transmitted across the network in cleartext.
5. Encrypted application data — all subsequent traffic is encrypted and integrity-protected with the derived session keys.

SET (Secure Electronic Transaction) is a protocol designed for online credit card payments. It uses dual signatures so the merchant sees the order details but not the full card number, and the bank sees the card number but not the order contents — neither party holds enough information alone to commit fraud. SET is largely replaced in practice by TLS combined with tokenisation (card numbers replaced with single-use tokens), but it remains in the FACT syllabus as a historically significant protocol.

VPN (Virtual Private Network) creates an encrypted tunnel so that traffic appears to come from the VPN server's IP rather than the user's actual IP. Types:

• IPSec VPN — Layer 3, encrypts all IP traffic (covered in Section 9.1). Standard for site-to-site corporate tunnels.
• SSL/TLS VPN — uses TLS (typically port 443) to tunnel traffic. Works through most firewalls since port 443 is rarely blocked. Used in most remote-access VPN products (Cisco AnyConnect, Palo Alto GlobalProtect).
• WireGuard — a modern, lightweight protocol (~4 000 lines of code); uses Curve25519 key exchange and ChaCha20 encryption; gaining adoption for performance and auditability.

Split tunnelling is a VPN option where only traffic destined for the corporate network goes through the VPN tunnel; all other internet traffic goes directly from the device to the internet, bypassing the VPN. Forensic implication: VPN server logs only contain corporate-destined traffic — the user's general internet activity during the same session is invisible to corporate monitoring. An investigator cannot rely on VPN logs alone to reconstruct full remote-employee activity.