My EVTX triage rules missed 4 of 11 real attack samples. Here's why.
When an analyst gets .evtx files from a host, the first questions are what happened, when, and which technique. Scrolling Event Viewer doesn't answer them. So I wrote evtx-triage: it parses Security, Sysmon and PowerShell logs, runs 14 rules mapped to MITRE ATT&CK, and writes a timeline CSV plus a findings report.
What the rules look at
- Security: 4625, 4771 and 4768 for spray and brute force, 1102 for log clearing, 4720 for new accounts, 5145 for PsExec pipes, 4688 for process creation.
- Sysmon: event 1 for LOLBins, WMI and schtasks, 10 for LSASS access, 13 for COM hijacking, 17/18 for named pipes.
- PowerShell 4104 script-block logging, matched on keywords.
My unit tests used hand-built XML events and all passed. Then I scored the tool against 11 public samples from EVTX-ATTACK-SAMPLES (203 records), each with a known technique.
First run: 7 of 11
All four misses came from the real data looking different from what I'd assumed:
- LSASS access. Sysmon writes
GrantedAccessas0x00001010. My allow-list said0x1010. I now compare hex values as numbers. - Kerberos spray. Same zero-padding on
Status(0x00000018). The spray also showed up as 4768 with status0x6, unknown principal, which is user enumeration I hadn't counted. After the fix it found 9 accounts tried from 172.16.66.1. - COM hijack. Sysmon logs
HKU\<SID>_CLASSES\CLSID\...\InprocServer32, not theSoftware\Classespath I was matching. I widened the regex. - WMI lateral movement. On older Windows, 4688 has no
ParentProcessName. I rebuild the parent by joiningProcessIdtoNewProcessId, which showsWmiPrvSE.exe → calc.exe.
Second run: 11 of 11
The 10 attack samples were flagged with the right technique, and the one benign logon sample produced no findings. Each fix has a regression test, and the suite is 7 passing tests.
The lesson I took: synthetic events test your logic, not your parser. Field formats differ between Windows versions and between Sysmon and the Security log, and you only find that out on real files. Eleven samples is also a small set, so I treat 11/11 as "no known misses", not as a detection rate.