Malware Forensics: Static, Dynamic, Sandbox and Memory Analysis

Your journey to becoming a forensic professional starts here.

Practice with national-level exam (FACT, FACT Plus, NET, CUET, etc.) mocks, learn from structured notes, and get your doubts solved in one place.

Start Free Mock Test Create Your Account

Malware Forensics: Static, Dynamic, Sandbox and Memory Analysis | ForensicSpot

The AIIMS Delhi ransomware incident of November 2022 took five of the hospital's main servers offline for nearly two weeks, disrupted patient registration and laboratory reporting for a flagship public hospital, and pulled in CERT-In, NIA, Delhi Police and the Indian Cyber Crime Coordination Centre at once. The publicly attributed family was a BlackCat/ALPHV variant, and a recurring observation in the after-action notes was that the affected systems had not been imaged in a way that preserved volatile state, so the malware-forensic team rebuilt the timeline from disk artefacts and backup metadata rather than from a clean memory image of the running infection. That gap is the practical reason this topic exists: the malware-forensic playbook has to be rehearsed before the incident, not assembled during it.

Malware forensics is four disciplines stitched together. Static analysis reads the binary without running it: PE headers, imports, strings, hashes, YARA matches. Dynamic analysis runs the binary in an instrumented host and watches behaviour: process tree, registry writes, network beacons, dropped files. Sandbox analysis runs the same exercise inside an isolated VM with telemetry baked in (Cuckoo, Any.Run, Joe Sandbox, Triage). Memory analysis reconstructs the state of an already-infected machine from a RAM image with Volatility 3 or MemProcFS, often catching what disk and sandbox both miss (injected code, hollowed processes, fileless payloads). This topic walks each discipline with Indian context, the tool versions in current use at CFSL Hyderabad and NFSU, and the CERT-In advisory and reporting workflow that the analysis feeds into.

Key terms

PE (Portable Executable) header: The Windows executable file format. Starts with an MZ DOS stub, points to a PE signature, then a COFF file header, an optional header (despite the name, mandatory in practice), and section headers (.text, .data, .rdata, .rsrc). Imports/exports tables and the resource directory are the analyst's primary entry points.
ImpHash: Mandiant's hash of the ordered list of imported functions in a PE file's import table. Survives recompilation that does not change imports; clusters malware families across builds. Implemented in pefile (Python) and Mandiant's original tool.
Entropy (Shannon): Per-byte information density of a file or section. Plain code sits around 5.5; packed or encrypted sections exceed 7.0. A PE with a high-entropy section that is not a known packer signature is almost always packed; the entropy line is the first static signal of obfuscation.
YARA: Pattern-matching language for malware classification. Rules combine string and byte-pattern conditions with metadata. Used by VirusTotal, every major sandbox, and most threat-intel platforms. Rule sets like YARA-Forge, Florian Roth's signature-base and the YARA hub at YARA-Rules are common starting points.
Volatility 3: Open-source memory forensics framework, Python 3 rewrite of Volatility 2. Symbol-based (PDB symbols on Windows) instead of profile-based, supports modern Windows 10/11 and Linux kernels, plugin ecosystem includes pslist, pstree, psxview, malfind, hollowfind, netscan, dlllist.
IoC (Indicator of Compromise): Observable artefact linked to malicious activity. File hashes (MD5, SHA-256, ImpHash, ssdeep, TLSH), IPs, domains, URLs, registry keys, mutex names, named pipes, scheduled task names, service names. Shared between organisations using STIX (data model) over TAXII (transport).

Malware taxonomy and the question of family

Naming a sample correctly halves the rest of the work.

A malware sample is rarely a single category. A modern banker is also a dropper, a downloader, a credential-stealer and often a coin-miner in a fallback branch. The taxonomy below is still useful because it pins the primary behaviour the analyst should look for first.

Family	Primary behaviour	Persistence tendency	Indian-case examples
Virus	Infects host files; spreads by file copy.	Patches existing binaries.	Largely historical; some classic Indian college-network outbreaks (2000s).
Worm	Self-propagating across networks.	Service or scheduled task on each new host.	WannaCry impact on Indian state agencies in 2017; Conficker in the late 2000s.
Trojan	Pretends to be benign; delivers payload.	Run key or service after first execution.	Bundle of cracked software from Indian forums; banking-app lookalikes on third-party Android stores.
Ransomware	Encrypts files, demands payment.	Service or scheduled task; sometimes none (single-shot detonators).

Static analysis: PE header, strings, hashes, entropy, YARA

Everything you can learn without pressing Run.

Static analysis costs nothing in execution risk. The trade-off is that obfuscated, packed or encrypted samples reveal less; the analyst still gets enough to plan dynamic analysis and to start clustering against known families.

The PE header is the first stop on Windows samples. PEview, PEStudio, CFF Explorer and Detect It Easy (DIE) all parse it. The analyst reads:

Compilation timestamp in the COFF file header. Often spoofed by malware authors, but a precisely-aligned timestamp at midnight UTC in a year that does not match the campaign is itself a tell.
Section headers. Standard sections are .text (code), .data, .rdata, .bss, .rsrc. Non-standard names (.UPX0, .UPX1, .nsp1, .vmp0) are packer signatures. A section marked executable that has a virtual size much larger than its raw size on disk is a packer-unpack stub.
Imports table. The functions the binary asks the loader to resolve. CreateRemoteThread, VirtualAllocEx and WriteProcessMemory together suggest process injection. WinHttpOpen and InternetReadFile suggest HTTP C2. CryptEncrypt, BCryptEncrypt, CryptGenKey together suggest ransomware crypto.
Exports table. DLLs and some EXEs export functions. An EXE with one named export "ServiceMain" is a Windows service; an EXE with exports "Run" or "Start" matching loader conventions is a stager.
Resources directory. Icons, manifests, version info, and embedded payloads. Malware regularly hides a second-stage payload in the .rsrc section, sometimes encrypted with a key in .text.

Strings extraction is the second stop. The default strings utility prints printable ASCII; strings -e l prints UTF-16LE which Windows malware uses for most user-visible strings. Florian Roth's floss (FireEye Labs Obfuscated String Solver) goes further, deobfuscating stack strings and simple XOR-encoded strings by running the binary symbolically. A high-value strings hit is a C2 domain, a ransom-note template, a debug PDB path that contains an attacker username, or an unusual user-agent.

Hashes are next, in three flavours.

MD5 and SHA-256 are the canonical file hashes. Identical bytes give identical hashes; a one-bit change defeats them. Useful for direct cross-reference against VirusTotal and CERT-In advisories.

Dynamic analysis and sandbox tiers

Run it, watch what it touches, write down everything.

Dynamic analysis observes behaviour as the sample executes. The instrumentation stack on a Windows analyst workstation is well-established:

Process Monitor (Procmon, Sysinternals) logs every file, registry, process and thread operation by every process. Filtered to the suspect process tree, it surfaces every dropped file, every registry write (Run keys, services, scheduled tasks), every spawned child process. Procmon traces are the loudest single signal in dynamic analysis.
Process Hacker is the modern Task Manager replacement. Lists processes with parent-child relationships, command lines, loaded modules, handles, threads, and (with the kernel driver) hidden processes. Right-click on a suspect process reveals strings in memory, which often exposes plaintext C2 URLs that the on-disk binary kept encrypted.
Wireshark captures network activity (see our Live Packet Capture topic for the capture-side detail).
INetSim is a Linux service that fakes the internet for the sandbox VM. Responds to DNS, HTTP, HTTPS, SMTP, FTP and IRC with plausible defaults, lets the malware proceed past its connectivity check, and logs every request as the C2 indicator set.
API Monitor / API hooks instruments user-mode API calls. Useful for tracking process-injection sequences (OpenProcess → VirtualAllocEx → WriteProcessMemory → CreateRemoteThread).
Sysmon with a sensible config (Olaf Hartong's modular config or SwiftOnSecurity's classic config) writes detailed process, network and file events to the Windows event log, which gives a parallel timeline that the sandbox can export.

The sandbox sits a tier above the analyst workstation: a fully-isolated VM with snapshot revert, full telemetry, and a controlled network. The current open-source standard is Cuckoo Sandbox (the maintained fork CAPE Sandbox in modern deployments), which orchestrates VM snapshots, drops the sample, runs it for a configured time, collects Procmon-equivalent telemetry, captures a memory image at the end, and writes a single JSON report with file, registry, network and process events.

Sandbox

Reverse engineering and unpacking

When the sandbox stops giving you new behaviour, open the binary.

Static and dynamic together answer the surface questions. The deeper questions (what algorithm is the ransomware using to encrypt, what is the format of the C2 protocol, what is the configuration block decoded by stage 2) need disassembly and debugging.

The disassembler choices in Indian labs:

IDA Pro is the long-standing commercial standard. Fast disassembly, scriptable in Python (IDAPython), decompiler (Hex-Rays) at extra cost. NFSU and several CFSL labs hold IDA Pro licences.
Ghidra is the NSA's open-source release. Free, with a built-in decompiler that is competitive with Hex-Rays for many architectures. The de facto default in academic and resource-constrained labs.
Radare2 / rizin / Cutter are the open-source CLI-and-GUI stack. Powerful, scriptable, smaller user base.
Binary Ninja is the modern commercial alternative; cheaper than IDA Pro, with a clean MLIL/HLIL pipeline.

The debugger choices:

x64dbg is the open-source Windows debugger of choice. Plugin ecosystem (Scylla for IAT reconstruction, SwissArmyKnife, ScyllaHide for anti-anti-debug).
WinDbg ships with the Windows SDK and is the only viable debugger for kernel-mode work and minidump analysis.
OllyDbg is older; still useful for 32-bit Windows malware that targets a Windows 7 era.
gdb is the canonical Linux debugger; with pwndbg, peda or gef plugins it is competitive with x64dbg for ELF samples.

Unpacking is the most common reverse-engineering subtask. The simple case is UPX, which has a public unpacker (upx -d). The harder case is custom packers, where the analyst sets a breakpoint at the original entry point of the unpacked code (the standard trick is to break on VirtualAlloc with EXECUTE_READWRITE protection and watch the unpacker write into the allocated region), then dumps the unpacked image and reconstructs the import address table with Scylla. Commercial packers like Themida and VMProtect virtualise the original code into a custom bytecode and present the analyst with a much harder problem; these are normally handled by family-specific scripts and by tracing rather than by unpacking.

Memory analysis with Volatility 3 and MemProcFS

The artefact the sandbox writes at the end is also the artefact the live host can give you.

Memory analysis is the only discipline that catches code which never lands on disk. Fileless malware writes its payload directly into another process's address space, and a disk forensic image shows nothing. A RAM image of the running infection shows everything.

Acquisition first. Windows tools: winpmem (Rekall), DumpIt (Comae), Magnet RAM Capture. Each produces a raw memory dump (.raw or .mem) or a Microsoft crash dump (.dmp). The acquisition should be performed from a verified clean tool on removable media; the output should be written to that same removable media to avoid touching the suspect disk. Linux: AVML (Microsoft, AppArmor-aware); LiME (LKM-based, traditional). macOS: OSXPmem (legacy), with modern macOS being harder due to SIP and lacking maintained tools for the latest releases.

Volatility 3 is the current analysis framework. The Python 3 rewrite of the original Volatility, symbol-driven rather than profile-driven. The relevant plugin set:

Plugin	Question answered	Notable on Indian incidents
windows.info	Which Windows build, kernel base, DTB.	Confirms the symbol package matches the dump; mismatched symbols silently produce wrong results.
windows.pslist	Live processes from EPROCESS active links.	Standard process list; missing the hidden ones by design.
windows.pstree

IoCs, persistence, network behaviour and the report to CERT-In

Findings only matter if they ship in a form other defenders can use.

Indicators of Compromise are the deliverables that travel beyond the original investigation. The accepted IoC types and their canonical formats:

File hashes: MD5, SHA-1, SHA-256, ImpHash, ssdeep, TLSH. Always SHA-256 as the primary; the others corroborate.
Network indicators: IPv4/IPv6, FQDNs, URLs, URI patterns, JA3/JA3S fingerprints (cross-link to our Live Packet Capture topic).
Host indicators: registry keys with values, mutex names, named-pipe names, scheduled-task names, service names, file paths.
Email indicators: sender addresses, subject patterns, attachment hashes.
YARA rules as a transferable detection artefact.

STIX (Structured Threat Information eXpression) is the data model that wraps these into objects (Indicator, Malware, ThreatActor, AttackPattern, Relationship). TAXII (Trusted Automated eXchange of Intelligence Information) is the transport that carries STIX between organisations. CERT-In publishes advisories that include STIX bundles for the larger campaigns; an Indian SOC consumes these and contributes back when the lab produces new IoCs.

Persistence mechanisms are the host-indicator subset that an investigation almost always documents.

Mechanism	Where it lives	Detection artefact
Run / RunOnce keys	HKCU\Software\Microsoft\Windows\CurrentVersion\Run (per user); HKLM\Software\Microsoft\Windows\CurrentVersion\Run (machine).	Registry hive parse (RegRipper, Volatility printkey); often the first place malware drops.

Worked example

Pulling the timeline from a memory image of a compromised Indian PSU workstation

A state PSU workstation in Lucknow is suspected of beaconing to an external C2. The local SOC requests CFSL assistance. The responder arrives, runs DumpIt from a verified USB drive, writes a 16 GB memory image to a separate evidence drive, hashes it (SHA-256 a3...e9), and ships the image to the analyst.

Volatility 3 windows.info confirms the dump is Windows 10 22H2 build 19045 and the symbol package matches. windows.pstree shows winword.exe spawning powershell.exe spawning a child named "svchost.exe" running from C:\Users\Public\Documents\. The svchost.exe in a user directory is the first anomaly.

windows.malfind on the suspect svchost.exe finds an executable, writable page at 0x00000000020A0000 with a recognisable PE header in memory but no on-disk file backing. windows.dumpfiles --pid <pid> writes the resident image; static analysis on the dump shows it is a stripped Cobalt Strike beacon, with the configuration block decoded to a C2 URL https://203.0.113.42/cm.

windows.netscan confirms an established TCP/443 connection from the suspect svchost.exe to 203.0.113.42, established 18 minutes before the memory acquisition. windows.cmdline shows the parent powershell.exe was launched with -EncodedCommand and a base64 blob; decoding the blob reveals a one-line downloader against the same C2.

The persistence is a scheduled task named "MicrosoftEdgeUpdateTaskMachineUA" stored in C:\Windows\System32\Tasks; the XML contains a base64 PowerShell command identical to the captured cmdline. The Run key and WMI subscription paths are clean.

The IoCs produced: SHA-256 of the dumped beacon, ImpHash, the C2 IP 203.0.113.42, the JA3 fingerprint extracted from the matching Zeek ssl.log on the SOC's perimeter, the scheduled-task name, and the named pipe that the beacon opened locally (\\.\pipe\msagent_xx). The bundle is wrapped in STIX, signed by the analyst, and pushed to CERT-In's advisory channel. The Section 63 BSA certificate is drawn over the memory image, the Volatility 3 reports, and the dumped beacon binary. The original workstation is preserved offline pending further trial-court direction.

Practice

Question 1 of 5· 0 answered

Which static-analysis signal most reliably indicates a packed or encrypted PE section?

Frequently asked questions

How do Indian forensic labs handle malware samples to avoid accidental infection of the lab network?

The convention is a strictly air-gapped malware-handling VLAN, an analyst workstation that double-boots into an analysis OS with no domain join, and physical USB-only transfer of samples in password-protected ZIPs (password 'infected'). The sandbox VMs run on a hypervisor with snapshot revert; the only outbound connectivity is to a controlled INetSim host. CFSL Hyderabad and NFSU labs additionally maintain a sample archive with read-only mounts and per-sample SHA-256 verification on every access.

Volatility 3 or Volatility 2 in 2026, and why?

Volatility 3 for any modern incident. Volatility 2 was profile-based and lagged behind Windows 10 and Windows 11 builds; Volatility 3 uses symbol files (PDB metadata or generated ISF files) and supports current builds. The plugin set has caught up for the common questions (pslist, pstree, psxview, malfind, netscan, dlllist, filescan). Some long-tail Volatility 2 plugins do not yet have Volatility 3 equivalents, in which case run both.

What is the difference between an ImpHash and a fuzzy hash like ssdeep when clustering malware?

ImpHash hashes the ordered list of imported API functions. It survives recompilation that does not change imports and is excellent for clustering builds of the same family that share a code base. ssdeep and TLSH are content-based fuzzy hashes that produce similar outputs for similar files; they survive byte-level changes (NOP padding, recompilation, light obfuscation) up to a point. Use ImpHash first because it is faster and more specific; fall back to ssdeep/TLSH when imports have been hidden or dynamically resolved.

Should an Indian analyst always upload an unknown sample to VirusTotal?

Be cautious. VirusTotal is public; once a sample is uploaded, any subscriber can see it and the corresponding hash becomes discoverable. For an Indian incident with operational sensitivity (a state agency, an active intrusion, a campaign-specific lure) public upload tips off the attacker. Use a private intelligence platform (Hybrid Analysis private upload, Joe Sandbox private, in-house Cuckoo) for initial analysis; share IoCs through CERT-In once the operational case is closed.

rule CobaltStrike_DefaultBeacon_Config {
    meta:
        author = "ForensicSpot"
        family = "Cobalt Strike"
        reference = "Sentinel One default profile"
    strings:
        $sleep = { 69 68 69 68 69 68 69 68 }
        $sig1 = "%s as %s\\%s: %d" wide
        $sig2 = "ReflectiveLoader" ascii
    condition:
        uint16(0) == 0x5A4D and 2 of them
}

Cuckoo / CAPE	Self-hosted, open source	Free, customisable, integrates with INetSim, supports Windows/Linux/macOS/Android VMs.	Standard at NFSU, IIIT Hyderabad, several state cyber-forensic units.
Any.Run	Cloud, interactive	Real-time UI, analyst clicks through prompts as the malware runs; excellent for documents with macros.	Used by NPCI's FRM and several Indian private SOCs for fast triage.
Hybrid Analysis (Falcon Sandbox)	Cloud, CrowdStrike	Strong YARA + ML scoring, large historical corpus.	Cross-referenced by CERT-In advisories; not a primary detonation lab.
Joe Sandbox	Cloud or on-premises commercial	Detailed behavioural graphs, strong anti-evasion.	Procured by some Indian central agencies.
Triage (Recorded Future / Hatching)	Cloud	Massive throughput, family detection, IoC extraction.	Used by Indian financial-sector CSIRT-Fin for batch detonation.
MalwareBazaar / VirusTotal Sandbox	Cloud	Free, broad family coverage; less depth.	Public, used for first-look triage.