Independent defensive-security research
Can AI reliably distinguish a new security problem from a duplicate, recurrence, or already-remediated incident?
Defensive Drift is building a reproducible benchmark and evaluation pipeline for security-drift reconciliation across configuration evidence, historical incidents, scanner findings, logs, and remediation records.
Research integrity: no performance claims are published until they are measured against the frozen benchmark and linked to reproducible run artifacts.
Research question
Evidence-grounded defensive triage, not autonomous remediation.
Modern security evidence is fragmented across configuration snapshots, infrastructure-as-code, scanners, logs, deployment history, prior incidents, and operational notes. The project measures whether AI can reconcile those artifacts without hiding genuinely new or unresolved problems inside a false duplicate.
The benchmark requires each method to classify an observation as NEW, DUPLICATE, RECURRENCE, RELATED_BUT_DISTINCT, or INSUFFICIENT_EVIDENCE, while grounding its conclusion in the supplied evidence.
Frozen measurement contract
Success means learning where AI helps — and where it fails.
Duplicate precision & recall
Measures both unsafe over-merging and alert-fatigue-causing misses.
Novel-issue recall
Measures whether new or materially distinct conditions remain visible for review.
Dangerous false-duplicate rate
Primary security-safety metric for incorrect closure-like classifications.
Evidence grounding
Checks whether cited evidence actually exists and materially supports the conclusion.
Remediation accuracy
Tests whether models correctly distinguish detected, mitigated, remediated, recurred, unresolved, and unknown states.
Cost, latency & consistency
Compares practical operating cost, response time, and repeated-trial stability.
35-day pre-grant sprint
Five weekly gates. Every gate ends in evidence.
Research design frozen
Freeze taxonomy, ground-truth rules, metrics, sanitization boundaries, and experiment protocol before final model evaluation.
Benchmark v0.1 frozen
Produce at least 50 human-adjudicated, schema-valid cases with a stretch target of 75.
Conventional baselines complete
Run normalized lexical, TF-IDF/similarity, and embedding baselines on the frozen benchmark.
AI evaluation complete
Compare OpenAI and practical open-weight approaches, including repeated high-risk trials, cost, latency, and failure analysis.
Grant-ready evidence package
Publish methodology, preliminary results, reproducibility artifacts, failure catalog, research brief, and the final grant evidence matrix.
Current status
Day 1: research design in progress.
The sprint began on August 29, 2026. The immediate P0 work is defining the incident schema and ground-truth schema. Measured benchmark and model results will appear here only after the corresponding methodology gates are complete.
Public benefit
Built to be inspected, reproduced, and challenged.
Defensive Drift is intended to release a safe public benchmark subset, schemas, evaluator code, methodology, measured results, representative failure cases, and reproducibility guidance. Private operational records are not automatically public research data; public artifacts must pass explicit sanitization and confidentiality review.