Skip to content

Network Protocols and Traffic Interpretation

Investigators reading packet captures or network logs must understand the structure of TCP/IP, HTTP, SMTP, and related protocols to identify anomalous behaviour and extract evidence. This topic covers protocol layers, header fields with investigative significance, connection states, and timing patterns that distinguish normal traffic from intrusion or data exfiltration.

Last updated:

Share

Network protocols are the formal rules that govern how devices exchange data across a network. For a cyber investigator, understanding those rules is not optional background knowledge: it is the toolkit for reading packet captures, interpreting logs, and identifying the traffic patterns that distinguish a normal business session from an intrusion, a data exfiltration attempt, or command-and-control communication. The Internet Protocol stack organises communication into layers, each with its own header fields. Those header fields record source and destination addresses, port numbers, sequence numbers, timestamps, flags, and protocol-specific metadata, all of which carry investigative value when read in context.

The protocols most frequently encountered in investigations are TCP/IP at the transport and network layers, DNS for name resolution, HTTP and HTTPS for web traffic, SMTP, IMAP, and POP3 for email, and FTP or SMB for file transfer. Each protocol has characteristic behaviours that investigators learn to recognise: a normal TCP three-way handshake looks different from a SYN scan, a legitimate DNS query looks different from DNS tunnelling, and benign HTTP traffic has a different timing signature from automated credential-stuffing. Reading traffic is a skill built on knowing what normal looks like first.

Traffic interpretation intersects with legal authority. Capturing packets in transit requires lawful interception authority under national law. Analysing stored logs or captures already collected by a network owner during an incident sits in a different legal category than live interception. Investigators must know which regime applies before they begin, both to ensure admissibility and to protect the rights of persons whose communications may appear in the data. Legislation such as India's Information Technology Act 2000 Section 69, the US Electronic Communications Privacy Act, and the EU's ePrivacy Directive each set different thresholds, and international investigations must satisfy the requirements of every jurisdiction whose nationals' traffic appears in the capture.

LayerProtocol examplesKey investigative fieldsApplicationHTTP, HTTPS, DNS, SMTP, FTP,SMBURL, Host header, User-Agent, From/To fields, querynames, file namesTransportTCP, UDPSource port, dest port, sequence numbers, TCP flags(SYN/ACK/RST/FIN), checksumInternetIPv4, IPv6, ICMPSource IP, destination IP, TTL (hop count),protocol number, fragmentation flagsLinkEthernet, Wi-Fi (802.11)Source MAC, destination MAC, VLAN tag (localsegment only, not end-to-end)Layer nameProtocol examplesInvestigative fields
Each TCP/IP layer contributes distinct header fields: the investigative fields in the right column are what investigators actually read in a packet capture to establish who, where, and how.

By the end of this topic you will be able to:

  • Identify the layers of the TCP/IP model and name the key header fields at each layer that carry investigative value.
  • Interpret TCP flags and connection states to distinguish normal handshakes from scanning, session hijacking, and denial-of-service patterns.
  • Explain the investigative significance of HTTP headers, SMTP Received chains, and DNS query-response pairs in evidence extraction.
  • Describe timing patterns and volume anomalies that indicate data exfiltration, beaconing malware, and brute-force attacks.
  • Outline the legal framework governing packet capture and log analysis in at least two jurisdictions.
Key terms
Packet capture (PCAP)
A file format that stores raw network frames as captured from a network interface. Tools such as Wireshark and tcpdump produce PCAP files. Each frame contains full headers and payload, making PCAP the most granular form of network evidence.
TCP three-way handshake
The sequence SYN, SYN-ACK, ACK that establishes a TCP connection. The initiating host sends SYN; the responding host replies SYN-ACK; the initiator completes with ACK. Any deviation from this sequence indicates scanning, filtering, or interference.
Port number
A 16-bit integer in the TCP or UDP header that identifies the application-layer service at each endpoint. Well-known ports are assigned by IANA: HTTP is 80, HTTPS is 443, SMTP is 25, DNS is 53. Traffic on unexpected port numbers is a common indicator of tunnelling or evasion.
Beaconing
Periodic outbound connections from a compromised host to a command-and-control server, typically at regular intervals. The regularity of the interval, measured in seconds or minutes, distinguishes malware beaconing from human browsing patterns and is detectable through timing analysis of connection logs.
Network flow (NetFlow/IPFIX)
A summary record of a network conversation, storing source IP, destination IP, source port, destination port, protocol, byte count, and timestamps, without the packet payload. Flow records are less granular than PCAP but are retained for longer periods and cover higher-volume networks.
DNS tunnelling
Encoding data inside DNS queries and responses to exfiltrate information or carry command-and-control traffic through a network that permits DNS but blocks other protocols. Indicators include unusually long domain names, high-entropy subdomains, and query volumes far above what name resolution alone would generate.

The TCP/IP model and its investigative layers

The TCP/IP model organises network communication into four layers: link, internet, transport, and application. Each layer adds its own header to the data being transmitted, and each header contains fields that matter to investigators. Understanding which layer holds which information prevents the common mistake of looking for an answer in the wrong header.

LayerProtocol examplesKey investigative fields
LinkEthernet, Wi-Fi (802.11)MAC address (source and destination), VLAN tag
InternetIPv4, IPv6, ICMPSource IP, destination IP, TTL, protocol number, fragmentation flags
TransportTCP, UDPSource port, destination port, sequence/acknowledgment numbers, TCP flags, checksum
ApplicationHTTP, DNS, SMTP, FTP, SMBProtocol-specific headers, URL, From/To fields, query names, file names

The IP Time to Live (TTL) field deserves particular attention. Each router along a path decrements the TTL by one; when it reaches zero the packet is dropped and an ICMP Time Exceeded message is sent back to the source. Investigators use TTL values observed at the capture point to estimate how many hops the packet has traversed. An attacker spoofing a source IP cannot control the TTL that reaches the capture point, so an unexpectedly low TTL on traffic claiming to originate from a local subnet is a red flag.

MAC addresses operate only within a single network segment. They are not preserved end-to-end across routed networks. This limits their value in wide-area investigations but makes them useful in local-network investigations: the MAC address of the device that connected to a switch port can identify the specific machine, and ARP tables can map MACs to IPs at the time of the incident.

TCP connection states and their investigative meaning

TCP is a stateful protocol. A connection moves through defined states from establishment to teardown, and each state transition leaves a record in packet captures and in the connection tables maintained by operating systems and firewalls. Investigators read these states to reconstruct what happened and when.

The TCP header contains six control flags: URG, ACK, PSH, RST, SYN, and FIN. Of these, SYN, ACK, RST, and FIN are the most frequently significant in investigations. A SYN with no reply indicates the destination port is filtered. A SYN followed immediately by RST indicates the port is closed. A SYN followed by SYN-ACK and then ACK indicates a successful connection. A stream of SYN packets from one source to many destination ports with no follow-through is a port scan.

TCP sequence numbers are 32-bit counters that track bytes transferred in each direction. In a legitimate connection, sequence numbers increment predictably. Session hijacking attempts involve an attacker injecting packets with forged sequence numbers that fall within the expected window. Detecting this requires comparing observed sequence numbers against what the connection state table predicts. Modern operating systems use randomised initial sequence numbers (ISNs) to make blind hijacking harder, but the technique was historically exploited and sequence number analysis remains part of post-incident review.

A CLOSE_WAIT state that persists for hours indicates a connection where the remote side has sent FIN but the local application has not responded. Large numbers of connections in TIME_WAIT or SYN_RECEIVED states are indicators of specific attack types: TIME_WAIT accumulation can result from rapid connection cycling, and SYN_RECEIVED accumulation is the signature of a SYN flood denial-of-service attack.

HTTP and HTTPS traffic interpretation

HTTP is the primary application-layer protocol for web traffic and the most common channel for both legitimate user activity and attack traffic. An HTTP request consists of a method line (GET, POST, PUT, DELETE, and others), a set of headers, and an optional body. An HTTP response consists of a status line, response headers, and the response body. Each part carries information that investigators use to reconstruct user activity and attacker behaviour.

The most investigatively significant HTTP request headers are: Host (the intended virtual host), User-Agent (the client software, which attackers often set to mimic browsers or leave as a tool signature), Referer (the URL from which the request originated, useful for reconstructing navigation paths), and Authorization or Cookie (which carry credentials or session tokens). Response headers such as Set-Cookie and Location (for redirects) are equally important: tracking cookies can link multiple sessions to the same actor, and redirect chains can reveal infrastructure used in phishing or traffic distribution.

HTTPS encrypts the HTTP payload using TLS, so the request headers and body are not visible in a packet capture without a decryption key or a proxy performing TLS inspection. However, several fields remain visible even in encrypted traffic: the destination IP address, the destination port, the TLS Server Name Indication (SNI) extension in the ClientHello message, and the certificate presented by the server. SNI reveals the domain name the client intended to connect to even when the payload is encrypted. Certificate analysis can reveal self-signed certificates used by malware command-and-control servers, expired certificates, and certificates with suspicious subject names.

Email protocols and the SMTP Received chain

Email moves through multiple mail transfer agents (MTAs) from sender to recipient. Each MTA prepends a Received header to the message that records the hostname of the server, the IP address of the system it accepted the connection from, and a timestamp. Reading these headers from bottom to top (oldest first) traces the message's path from its point of origin to the recipient's mailbox.

The From and Reply-To headers are supplied by the sending client and can be forged. The Received chain cannot be forged from the outside: each MTA adds its own Received header and the recipient's mail server adds the final one. The first Received header (lowest in the stack) contains the IP address that connected to the first MTA in the chain. That IP is the closest thing to the sender's actual origin, though it may be a VPN, proxy, or compromised relay rather than the attacker's own address.

Three authentication mechanisms are used to reduce email spoofing: SPF (Sender Policy Framework) lists authorised sending IPs for a domain in DNS TXT records; DKIM (DomainKeys Identified Mail) adds a cryptographic signature to the message headers that the receiving server verifies against a public key in DNS; and DMARC (Domain-based Message Authentication, Reporting and Conformance) specifies how a receiving server should handle messages that fail SPF or DKIM and where to send reports. Investigation of phishing emails includes checking whether these mechanisms passed or failed and which part of the chain produced the failure.

IMAP (port 143 or 993 with TLS) and POP3 (port 110 or 995 with TLS) are the client-side retrieval protocols. Log analysis of IMAP connections can show when a mailbox was accessed, from which IP addresses, and how many messages were downloaded in a session. A sudden access from an unfamiliar geographic location followed by a large download is a characteristic pattern of business email compromise account takeover.

Anomaly patterns: exfiltration, beaconing, and brute force

Traffic anomalies are deviations from an established baseline. Investigators detect them by comparing current or historical traffic metrics against what the network normally looks like. Three patterns appear most frequently in cyber investigations: data exfiltration, command-and-control beaconing, and brute-force credential attacks.

Data exfiltration produces unusual outbound volume from a host that normally generates little outbound traffic, transfers to external IP addresses not seen before in the network's history, or activity outside normal business hours. Exfiltration over encrypted channels such as HTTPS or DNS is designed to blend in with legitimate traffic, but volume anomalies remain visible even when payload is encrypted. An internal server that normally transfers 50 MB outbound per day and suddenly transfers 4 GB overnight warrants investigation regardless of the protocol.

Beaconing is the periodic check-in of malware to a command-and-control server. The key distinguishing feature is regularity: a compromised host making outbound HTTP or DNS connections every 60 seconds, or every 300 seconds, produces a timing histogram that looks nothing like human browsing. Tools such as RITA (Real Intelligence Threat Analytics) and commercial SIEM platforms flag regular-interval connections automatically. Investigators also look at beacon jitter: sophisticated malware adds random delay to avoid exact-interval detection, but even jittered beacons cluster tightly in a timing histogram compared to human traffic.

Brute-force attacks against SSH (port 22), RDP (port 3389), SMTP authentication, or web login forms produce high volumes of connection attempts to a single service in rapid succession. The pattern is distinctive in firewall and authentication logs: thousands of failed authentication attempts from one or a few source IPs, often cycling through common usernames. Distributed brute-force attacks spread attempts across many source IPs to evade rate limits but still produce abnormally high authentication failure rates on the target. Investigators correlate authentication logs with connection logs to distinguish distributed attacks from normal failed logins.

Check your understanding
Question 1 of 4· 0 answered

An investigator filters a PCAP and sees thousands of SYN packets from one source IP to many destination ports on a single host, with the majority of responses being RST. What activity does this indicate?

Key Takeaways

  • The TCP/IP model's four layers each contribute distinct header fields to a packet: MAC addresses at the link layer, IP addresses and TTL at the internet layer, port numbers and flags at the transport layer, and protocol-specific metadata at the application layer.
  • TCP flags and connection states reveal attack behaviour: SYN-only traffic to many ports indicates scanning, SYN floods produce SYN_RECEIVED accumulation, and RST storms can mark scanning or active interference with sessions.
  • HTTP Host and User-Agent headers, the SMTP Received chain, and TLS SNI remain investigatively valuable even when payload content is encrypted or forged, because they record connection intent and path in ways attackers cannot fully control.
  • Three traffic patterns dominate cyber investigation: volume spikes from low-traffic hosts indicate exfiltration; regular-interval small connections indicate beaconing; high-rate authentication failures indicate brute-force attacks.
  • Traffic capture authority differs by context and jurisdiction. In India, Section 69 of the IT Act 2000 governs lawful interception; in the US, the Electronic Communications Privacy Act applies; in the EU, the ePrivacy Directive and GDPR constrain how traffic data may be collected and retained.
What is the investigative significance of TCP flags in a packet capture?
TCP flags control connection state and reveal attacker behaviour. A SYN flood shows many SYN packets with no ACK replies, indicating a denial-of-service attempt. An RST storm can signal port scanning. A FIN or RST on an established session pinpoints session termination time, which matters for establishing a timeline. Investigators also look for unusual flag combinations such as SYN-FIN or NULL scans used by reconnaissance tools like Nmap.
How does the HTTP Host header assist a network forensics investigation?
The Host header in an HTTP/1.1 request names the virtual host the client intended to reach. On a shared server hosting many domains, the Host header identifies which site a suspect visited even when all traffic goes to the same IP address. It is also used to detect domain-fronting techniques where the outer TLS SNI differs from the inner HTTP Host, which is a common method for tunnelling command-and-control traffic through legitimate CDN infrastructure.
Why are SMTP header fields valuable in email forensics?
Each mail transfer agent that processes a message prepends a Received header listing its own hostname, the IP address it accepted the connection from, and a timestamp. Reading these headers in reverse order traces the message path from sender to recipient. Forged From and Reply-To fields do not affect the Received chain, so investigators compare the claimed sender domain in the From header against the actual IP in the first Received header to detect spoofing.
What is a baseline and why does it matter for traffic anomaly detection?
A baseline is a documented profile of normal traffic volume, protocol mix, connection patterns, and timing for a network or host during a defined reference period. Anomaly detection compares current traffic against that baseline to flag deviations. Without a baseline, investigators cannot distinguish a spike caused by a backup job from one caused by data exfiltration. Baselines must be recalibrated when the network changes, as a stale baseline generates excessive false positives.
What legal frameworks govern packet capture in network investigations?
In India, lawful interception is authorised under the Information Technology Act 2000, Section 69, which permits government agencies to intercept, monitor, or decrypt information with appropriate approval. The Bharatiya Nagarik Suraksha Sanhita 2023 governs search and seizure of electronic records. In the US, the Electronic Communications Privacy Act and the Pen Register Act regulate traffic interception. In the EU, the Directive on Attacks Against Information Systems and GDPR Article 6 constrain how captured data may be processed and retained.

Test yourself on Cyber Forensics with free, timed mocks.

Practice Cyber Forensics questions

Found this useful? Pass it along.

Share

Spotted an error in this page? Report a correction or read our editorial standards.

Your journey to becoming a forensic professional starts here.

Practice with mock tests, learn from structured notes, and get your questions answered by a global forensic community, all in one place.