SENTINEL

SOC triage and incident response.

94%

HELD-OUT ACCURACY

Sealed

EVALUATION SET

Pre-Pilot

STATUS

What It Is

SENTINEL tests classification accuracy on Security Operations Center alert triage and incident-response routing. The benchmark evaluates the system against a sealed held-out set of real-world SOC alerts, measuring the ability to correctly classify alerts as true positive, false positive, or requiring escalation.

The held-out evaluation set was sealed before any pipeline contact. The 94% accuracy figure represents classification performance on alerts the system has never seen during development or tuning.

Methodology

CorpusSealed held-out evaluation set

BaselinesBaseline comparison documented in repo

PipelineDeterministic classification, no LLM at runtime

AttributionPer-fix attribution in methodology arc

LimitationsSOAR auth gaps documented, pre-pilot status

ReproducibilityFull instructions in GitHub repo

Reproducibility

Evaluation setSealed held-out alerts
Seal methodSHA-256 in CHECKPOINT_RESULTS.md
Repogithub.com/jourdanlabs/benchmarks/sentinel

Limitations

SOAR auth unknowns. SOAR vendor adapters have documented gap files requiring sign-off before production deployment. Adapters are functional stubs.

Pre-pilot status. Currently in pre-pilot with a partner organization. No live production deployment yet.

Feedback loop thresholds. FP-rate heuristics are v0 placeholders and require verification before design partner exposure.