{
 "meta": {
  "id": "configurations",
  "title": "Every measured configuration on the MaliciousSkillBench source-disjoint test split",
  "sources": [
   "deterministic.html#1–#3",
   "judged.html#1–#2",
   "cascade.html#3, #6",
   "large-scale.html#1–#5, #9, #15–#16",
   "cross-tool.html (msb-source-disjoint tables)",
   "improvements.html#1–#3, #5, #9–#10",
   "recommendations.html#2",
   {
    "repo": "Vineethsain/skill-scanner-vs-skillspector",
    "path": "results/large-scale.json",
    "sha256": "e48f9cb342c05d2d13286c6b91c165411994d6febd71e14d1fe54c6f339d32b5"
   },
   {
    "repo": "Vineethsain/cisco-skill-scanner-judge-and-system-one-results",
    "path": "RESULTS.md",
    "sha256": "2503c65d7bda785a6b4ddcc7bb0e1bb56437a3a23c8f332ffab04b7f6414daa3"
   }
  ],
  "extracted_at": "2026-10-16T14:34:41+00:00",
  "notes": [
   "One row per configuration. metrics are fractions; tp/fp/fn/tn where the page or JSON gives counts.",
   "lens: high_plus = block at HIGH+ (the shipped gate), medium_plus = review at MEDIUM+, any_intervention = confirm or block counts as a catch, any_finding = any finding at all, shipped_gate = each tool's own gate, flag = classifier threshold.",
   "population is msb-source-disjoint (839 malicious, 545 benign, 1,384 records) unless the string says otherwise; n is the scored count.",
   "hero=true marks the configurations the chapter-1 ROC scatter shows by default; the rest are available behind a filter.",
   "superseded=true: the held-out-half cascade whose threshold was chosen on test members.",
   "variant records which page/run a duplicate measurement came from; the README lists the conflicts.",
   "The index.html KPI tiles are in `index_kpis` with the table each one is backed by; the ninth tile (System One value ratio) is dropped per owner decision 3.",
   "superseded=true also marks the core_judge_meta / core_meta arms, in which the meta-judge was a schema-bug no-op (owner decision 1)."
  ],
  "page_sha256": {
   "deterministic.html": "752550a10cd6b5f5a0c465f1de4da1168521cefc519bedba7cd5c006f67e4cc4",
   "judged.html": "c8a6f6c0912a7791db4b80f3195dd316bc5603fbae3ecc0ad9297232724f0ebd",
   "cascade.html": "c1a4d8157eba904912e05da37e55dd3c9a4abc313064bd6dc0180ecce526bdc1",
   "large-scale.html": "2101600e4bc7833d4222bc8f066a548661443464a1ee4981cac5d07372d25a62",
   "cross-tool.html": "e7f8433087a55e382dc51895d350412a45ce19124d12beda376a47a1d55509aa",
   "improvements.html": "dbb68812c9eca9d368773fb520761af6c33836e65c5b7bcc35d84a0e642fcfb6",
   "recommendations.html": "cbb0c3dbc9b80000a67a18f13ae85a94a88827748e65758fc04f17fb9a8ce2da"
  }
 },
 "data": {
  "configurations": [
   {
    "id": "rules-shipped-gate",
    "label": "Rules alone, shipped (block at HIGH+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0775,
     "precision": 0.6075,
     "f1": 0.1374,
     "fpr": 0.0771,
     "tp": 65,
     "fp": 42,
     "fn": 774,
     "tn": 503
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": {
     "seconds_per_skill": 0.014,
     "usd_per_corpus": 0.0,
     "source": "RESULTS.md cost table (dataset)"
    },
    "source": "deterministic.html#1, #2",
    "notes": null
   },
   {
    "id": "rules-shipped-medium",
    "label": "Rules alone, shipped (review at MEDIUM+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0799,
     "precision": 0.5929,
     "f1": 0.1408,
     "fpr": 0.0844
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "deterministic.html#3",
    "notes": null
   },
   {
    "id": "rules-shipped-medium_plus-ab",
    "label": "Rules alone, shipped tree 5b696a1 (review at MEDIUM+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07985697258641239,
     "precision": 0.5929203539823009,
     "f1": 0.14075630252100838,
     "fpr": 0.08440366972477065,
     "tp": 67,
     "fp": 46,
     "fn": 772,
     "tn": 499
    },
    "hero": false,
    "superseded": false,
    "variant": "static core arm, counts from labelled_ab",
    "cost": null,
    "source": "results/large-scale.json full.labelled_ab.corpora.msb-source-disjoint, large-scale.html#11",
    "notes": "recall/fpr/precision/F1 derived from tp/fp/positives/negatives; the page prints recall and FPR only."
   },
   {
    "id": "rules-final-medium_plus-ab",
    "label": "Rules alone, final tree 9c08673 (review at MEDIUM+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07985697258641239,
     "precision": 0.7444444444444445,
     "f1": 0.1442411194833154,
     "fpr": 0.04220183486238532,
     "tp": 67,
     "fp": 23,
     "fn": 772,
     "tn": 522
    },
    "hero": true,
    "superseded": false,
    "variant": "static core arm, counts from labelled_ab",
    "cost": null,
    "source": "results/large-scale.json full.labelled_ab.corpora.msb-source-disjoint, large-scale.html#11",
    "notes": "recall/fpr/precision/F1 derived from tp/fp/positives/negatives; the page prints recall and FPR only."
   },
   {
    "id": "rules-shipped-high_plus-ab",
    "label": "Rules alone, shipped tree 5b696a1 (block at HIGH+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07747318235995232,
     "precision": 0.6074766355140186,
     "f1": 0.13742071881606763,
     "fpr": 0.07706422018348624,
     "tp": 65,
     "fp": 42,
     "fn": 774,
     "tn": 503
    },
    "hero": false,
    "superseded": false,
    "variant": "static core arm, counts from labelled_ab",
    "cost": null,
    "source": "results/large-scale.json full.labelled_ab.corpora.msb-source-disjoint",
    "notes": "recall/fpr/precision/F1 derived from tp/fp/positives/negatives; the page prints recall and FPR only."
   },
   {
    "id": "rules-final-high_plus-ab",
    "label": "Rules alone, final tree 9c08673 (block at HIGH+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07747318235995232,
     "precision": 0.7471264367816092,
     "f1": 0.14038876889848811,
     "fpr": 0.04036697247706422,
     "tp": 65,
     "fp": 22,
     "fn": 774,
     "tn": 523
    },
    "hero": false,
    "superseded": false,
    "variant": "static core arm, counts from labelled_ab",
    "cost": null,
    "source": "results/large-scale.json full.labelled_ab.corpora.msb-source-disjoint",
    "notes": "recall/fpr/precision/F1 derived from tp/fp/positives/negatives; the page prints recall and FPR only."
   },
   {
    "id": "rules-demotion-0",
    "label": "Rules: Shipped, all rules",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07628128724672228,
     "precision": 0.5818181818181818,
     "f1": 0.13487881981032665,
     "fpr": 0.08440366972477065
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "results/large-scale.json deterministic.demotion, large-scale.html#9",
    "notes": "Parquet-store re-measurement of the shipped rules on the test split (F1 13.49%, not 13.74%)."
   },
   {
    "id": "rules-demotion-1",
    "label": "Rules: Demote CORRELATED_NETWORK_EXECUTION_FLOW",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07270560190703218,
     "precision": 0.7721518987341772,
     "f1": 0.1328976034858388,
     "fpr": 0.03302752293577982
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "results/large-scale.json deterministic.demotion, large-scale.html#9",
    "notes": "Parquet-store re-measurement of the shipped rules on the test split (F1 13.49%, not 13.74%)."
   },
   {
    "id": "rules-demotion-2",
    "label": "Rules: Also demote PIPELINE_TAINT_FLOW",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.06197854588796186,
     "precision": 0.8,
     "f1": 0.11504424778761063,
     "fpr": 0.023853211009174313
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "results/large-scale.json deterministic.demotion, large-scale.html#9",
    "notes": "Parquet-store re-measurement of the shipped rules on the test split (F1 13.49%, not 13.74%)."
   },
   {
    "id": "arm-core_only",
    "label": "core_only (shipped prompt, block at HIGH+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": "core_only",
    "prompt": "shipped",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0775,
     "precision": 0.6075,
     "f1": 0.1374,
     "fpr": 0.0771
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "judged.html#1",
    "notes": null
   },
   {
    "id": "arm-core_judge",
    "label": "core_judge (shipped prompt, block at HIGH+)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": "core_judge",
    "prompt": "shipped",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.2479,
     "precision": 0.7849,
     "f1": 0.3768,
     "fpr": 0.1046
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": {
     "usd_per_corpus": 1.0211,
     "seconds_per_skill": 2.377,
     "source": "RESULTS.md cost table (dataset)"
    },
    "source": "judged.html#1",
    "notes": null
   },
   {
    "id": "arm-core_judge_meta",
    "label": "core_judge_meta (shipped prompt, block at HIGH+)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": "core_judge_meta",
    "prompt": "shipped",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.2443,
     "precision": 0.7824,
     "f1": 0.3724,
     "fpr": 0.1046
    },
    "hero": false,
    "superseded": true,
    "variant": null,
    "cost": {
     "usd_per_corpus": 1.8454,
     "seconds_per_skill": 2.377,
     "source": "RESULTS.md cost table (dataset)"
    },
    "source": "judged.html#1",
    "notes": "superseded: the meta-judge was a schema-bug no-op in this run (owner decision 1); see stages.json meta_judge."
   },
   {
    "id": "arm-core_meta",
    "label": "core_meta (shipped prompt, block at HIGH+)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": "core_meta",
    "prompt": "shipped",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0775,
     "precision": 0.6075,
     "f1": 0.1374,
     "fpr": 0.0771
    },
    "hero": false,
    "superseded": true,
    "variant": null,
    "cost": null,
    "source": "judged.html#1",
    "notes": "superseded: the meta-judge was a schema-bug no-op in this run (owner decision 1); see stages.json meta_judge."
   },
   {
    "id": "judge-shipped-judged-page",
    "label": "Judge alone, shipped prompt (judged.html, any-intervention MEDIUM+)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.4899,
     "precision": 0.8091,
     "f1": 0.6102,
     "fpr": 0.178
    },
    "hero": false,
    "superseded": false,
    "variant": "judged.html#2: FPR 17.80%",
    "cost": null,
    "source": "judged.html#2",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-medium_plus-skill-scanner",
    "label": "Cisco skill-scanner, judge on Gemma 4 26B (medium plus)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.4899,
     "precision": 0.8155,
     "f1": 0.6121,
     "fpr": 0.1761
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": null,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#2",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-medium_plus-skillspector",
    "label": "NVIDIA SkillSpector, judge on Gemma 4 26B (medium plus)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.7998,
     "precision": 0.6566,
     "f1": 0.7211,
     "fpr": 0.6648
    },
    "hero": true,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": 26.018,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#2",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-high_plus-skill-scanner",
    "label": "Cisco skill-scanner, judge on Gemma 4 26B (high plus)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.2527,
     "precision": 0.8,
     "f1": 0.3841,
     "fpr": 0.1004
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": null,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#3",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-high_plus-skillspector",
    "label": "NVIDIA SkillSpector, judge on Gemma 4 26B (high plus)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.7545,
     "precision": 0.6941,
     "f1": 0.723,
     "fpr": 0.5284
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": 26.018,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#3",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-shipped_gate-skill-scanner",
    "label": "Cisco skill-scanner, judge on Gemma 4 26B (shipped gate)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "shipped_gate",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.2527,
     "precision": 0.8,
     "f1": 0.3841,
     "fpr": 0.1004
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": null,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#4",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-shipped_gate-skillspector",
    "label": "NVIDIA SkillSpector, judge on Gemma 4 26B (shipped gate)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "shipped_gate",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.3886,
     "precision": 0.7443,
     "f1": 0.5106,
     "fpr": 0.2121
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": 26.018,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#4",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-any_finding-skill-scanner",
    "label": "Cisco skill-scanner, judge on Gemma 4 26B (any finding)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "any_finding",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.9261,
     "precision": 0.6186,
     "f1": 0.7418,
     "fpr": 0.9072
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": null,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#5",
    "notes": null
   },
   {
    "id": "ct-gemma-4-26b-any_finding-skillspector",
    "label": "NVIDIA SkillSpector, judge on Gemma 4 26B (any finding)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "gemma-4-26b",
    "arm": "gemma-4-26b",
    "prompt": null,
    "passes": null,
    "lens": "any_finding",
    "population": "msb-source-disjoint",
    "n": 1367,
    "metrics": {
     "recall": 0.8093,
     "precision": 0.6442,
     "f1": 0.7174,
     "fpr": 0.7102
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1367)",
    "cost": {
     "seconds_per_skill": 26.018,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#7"
    },
    "source": "cross-tool.html#5",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-medium_plus-skill-scanner",
    "label": "Cisco skill-scanner, judge on Claude Haiku 4.5 (medium plus)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.4577,
     "precision": 0.8196,
     "f1": 0.5874,
     "fpr": 0.1564
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322) — labelled '(core rules)' on the page but is the Haiku judge arm",
    "cost": {
     "seconds_per_skill": 13.763,
     "input_tokens_per_skill": 10111.937972768532,
     "output_tokens_per_skill": 1593.9417549167927,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#30",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-medium_plus-skillspector",
    "label": "NVIDIA SkillSpector, judge on Claude Haiku 4.5 (medium plus)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.8458,
     "precision": 0.6409,
     "f1": 0.7292,
     "fpr": 0.7355
    },
    "hero": true,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322)",
    "cost": {
     "seconds_per_skill": 39.585,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#30",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-high_plus-skill-scanner",
    "label": "Cisco skill-scanner, judge on Claude Haiku 4.5 (high plus)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.2687,
     "precision": 0.8606,
     "f1": 0.4095,
     "fpr": 0.0676
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322) — labelled '(core rules)' on the page but is the Haiku judge arm",
    "cost": {
     "seconds_per_skill": 13.763,
     "input_tokens_per_skill": 10111.937972768532,
     "output_tokens_per_skill": 1593.9417549167927,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#31",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-high_plus-skillspector",
    "label": "NVIDIA SkillSpector, judge on Claude Haiku 4.5 (high plus)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.745,
     "precision": 0.6745,
     "f1": 0.708,
     "fpr": 0.5579
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322)",
    "cost": {
     "seconds_per_skill": 39.585,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#31",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-shipped_gate-skill-scanner",
    "label": "Cisco skill-scanner, judge on Claude Haiku 4.5 (shipped gate)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "shipped_gate",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.2687,
     "precision": 0.8606,
     "f1": 0.4095,
     "fpr": 0.0676
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322) — labelled '(core rules)' on the page but is the Haiku judge arm",
    "cost": {
     "seconds_per_skill": 13.763,
     "input_tokens_per_skill": 10111.937972768532,
     "output_tokens_per_skill": 1593.9417549167927,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#32",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-shipped_gate-skillspector",
    "label": "NVIDIA SkillSpector, judge on Claude Haiku 4.5 (shipped gate)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "shipped_gate",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.5485,
     "precision": 0.7462,
     "f1": 0.6323,
     "fpr": 0.2896
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322)",
    "cost": {
     "seconds_per_skill": 39.585,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#32",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-any_finding-skill-scanner",
    "label": "Cisco skill-scanner, judge on Claude Haiku 4.5 (any finding)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "any_finding",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.7463,
     "precision": 0.5742,
     "f1": 0.649,
     "fpr": 0.8591
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322) — labelled '(core rules)' on the page but is the Haiku judge arm",
    "cost": {
     "seconds_per_skill": 13.763,
     "input_tokens_per_skill": 10111.937972768532,
     "output_tokens_per_skill": 1593.9417549167927,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#33",
    "notes": null
   },
   {
    "id": "ct-claude-haiku-4.5-any_finding-skillspector",
    "label": "NVIDIA SkillSpector, judge on Claude Haiku 4.5 (any finding)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": "claude-haiku-4.5",
    "arm": "claude-haiku-4.5",
    "prompt": null,
    "passes": null,
    "lens": "any_finding",
    "population": "msb-source-disjoint",
    "n": 1322,
    "metrics": {
     "recall": 0.8458,
     "precision": 0.6409,
     "f1": 0.7292,
     "fpr": 0.7355
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1322)",
    "cost": {
     "seconds_per_skill": 39.585,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#35"
    },
    "source": "cross-tool.html#33",
    "notes": null
   },
   {
    "id": "ct-static-medium_plus-skill-scanner",
    "label": "Cisco skill-scanner, static only (medium plus)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.0799,
     "precision": 0.6036,
     "f1": 0.1411,
     "fpr": 0.0832
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 0.015,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#44",
    "notes": null
   },
   {
    "id": "ct-static-medium_plus-skillspector",
    "label": "NVIDIA SkillSpector, static only (medium plus)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.3433,
     "precision": 0.5614,
     "f1": 0.426,
     "fpr": 0.4253
    },
    "hero": true,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 2.453,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#44",
    "notes": null
   },
   {
    "id": "ct-static-high_plus-skill-scanner",
    "label": "Cisco skill-scanner, static only (high plus)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.0775,
     "precision": 0.619,
     "f1": 0.1377,
     "fpr": 0.0756
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 0.015,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#45",
    "notes": null
   },
   {
    "id": "ct-static-high_plus-skillspector",
    "label": "NVIDIA SkillSpector, static only (high plus)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.1454,
     "precision": 0.601,
     "f1": 0.2342,
     "fpr": 0.1531
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 2.453,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#45",
    "notes": null
   },
   {
    "id": "ct-static-shipped_gate-skill-scanner",
    "label": "Cisco skill-scanner, static only (shipped gate)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "shipped_gate",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.0775,
     "precision": 0.619,
     "f1": 0.1377,
     "fpr": 0.0756
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 0.015,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#46",
    "notes": null
   },
   {
    "id": "ct-static-shipped_gate-skillspector",
    "label": "NVIDIA SkillSpector, static only (shipped gate)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "shipped_gate",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.0131,
     "precision": 0.5789,
     "f1": 0.0256,
     "fpr": 0.0151
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 2.453,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#46",
    "notes": null
   },
   {
    "id": "ct-static-any_finding-skill-scanner",
    "label": "Cisco skill-scanner, static only (any finding)",
    "group": "rules",
    "tool": "skill-scanner",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "any_finding",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.6055,
     "precision": 0.5248,
     "f1": 0.5623,
     "fpr": 0.8696
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 0.015,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#47",
    "notes": null
   },
   {
    "id": "ct-static-any_finding-skillspector",
    "label": "NVIDIA SkillSpector, static only (any finding)",
    "group": "skillspector",
    "tool": "skillspector",
    "model": null,
    "arm": "static",
    "prompt": null,
    "passes": null,
    "lens": "any_finding",
    "population": "msb-source-disjoint",
    "n": 1368,
    "metrics": {
     "recall": 0.3433,
     "precision": 0.5538,
     "f1": 0.4238,
     "fpr": 0.4386
    },
    "hero": false,
    "superseded": false,
    "variant": "cross-tool.html, records usable on both tools (n=1368)",
    "cost": {
     "seconds_per_skill": 2.453,
     "input_tokens_per_skill": null,
     "output_tokens_per_skill": null,
     "source": "cross-tool.html#49"
    },
    "source": "cross-tool.html#47",
    "notes": null
   },
   {
    "id": "judge-shipped-local-vllm",
    "label": "Judge alone, shipped prompt, Gemma 4 26B on local vLLM",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1376,
    "metrics": {
     "recall": 0.5288461538461539,
     "precision": 0.8163265306122449,
     "f1": 0.6418672501823487,
     "fpr": 0.18198529411764705
    },
    "hero": false,
    "superseded": false,
    "variant": "local vLLM, n=1376 after decoding fix",
    "cost": null,
    "source": "results/large-scale.json judge.local, large-scale.html#1",
    "notes": null
   },
   {
    "id": "judge-shipped-mantle",
    "label": "Judge alone, shipped prompt, Gemma 4 26B via Bedrock mantle",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.49,
     "precision": 0.815,
     "f1": 0.612,
     "fpr": 0.176
    },
    "hero": true,
    "superseded": false,
    "variant": "mantle (earlier run), FPR 17.6%",
    "cost": null,
    "source": "results/large-scale.json judge.mantle, large-scale.html#1, improvements.html#10",
    "notes": null
   },
   {
    "id": "heldout-judge_alone",
    "label": "Judge alone, held-out half",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (held-out half, 721 records)",
    "n": 721,
    "metrics": {
     "recall": 0.5105386416861827,
     "precision": 0.7622377622377622,
     "f1": 0.6115007012622721,
     "fpr": 0.23129251700680273
    },
    "hero": false,
    "superseded": true,
    "variant": null,
    "cost": null,
    "source": "results/large-scale.json cascade, large-scale.html#2",
    "notes": "Threshold selected on test members; superseded by the train/validation-selected screens."
   },
   {
    "id": "heldout-cascade",
    "label": "OpenJev prompt_injection screen (p ≥ 0.035, chosen on test half) then judge",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (held-out half, 721 records)",
    "n": 721,
    "metrics": {
     "recall": 0.4918032786885246,
     "precision": 0.860655737704918,
     "f1": 0.6259314456035767,
     "fpr": 0.11564625850340136
    },
    "hero": false,
    "superseded": true,
    "variant": null,
    "cost": null,
    "source": "results/large-scale.json cascade, large-scale.html#2",
    "notes": "Threshold selected on test members; superseded by the train/validation-selected screens."
   },
   {
    "id": "screen-new-none-nocap",
    "label": "Judge alone, new prompt",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.657074340527578,
     "precision": 0.8896103896103896,
     "f1": 0.7558620689655172,
     "fpr": 0.12477064220183487,
     "tp": 548,
     "fp": 68
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 1.0,
     "real_skill_judge_calls_share": 1.0,
     "real_skill_flag_rate": 0.1045
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.6242,
      0.6885
     ],
     "fpr": [
      0.0996,
      0.1552
     ]
    }
   },
   {
    "id": "screen-new-prompt_injection-nocap",
    "label": "OpenJev prompt_injection screen, then judge, new prompt",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.5947242206235012,
     "precision": 0.9612403100775194,
     "f1": 0.7348148148148148,
     "fpr": 0.03669724770642202,
     "tp": 496,
     "fp": 20
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5997099347353154,
     "real_skill_judge_calls_share": 0.079,
     "real_skill_flag_rate": 0.0315
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.561,
      0.6275
     ],
     "fpr": [
      0.0239,
      0.056
     ]
    }
   },
   {
    "id": "screen-new-logistic-nocap",
    "label": "OpenJev logistic screen (8 probes), then judge, new prompt",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.6127098321342925,
     "precision": 0.9751908396946565,
     "f1": 0.7525773195876289,
     "fpr": 0.023853211009174313,
     "tp": 511,
     "fp": 13
    },
    "hero": true,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5844815083393764,
     "real_skill_judge_calls_share": 0.0315,
     "real_skill_flag_rate": 0.0175
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.5792,
      0.6452
     ],
     "fpr": [
      0.014,
      0.0404
     ]
    }
   },
   {
    "id": "screen-new-none-low-confidence",
    "label": "Judge alone, new prompt, low-confidence findings capped at LOW",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.6211031175059952,
     "precision": 0.9008695652173913,
     "f1": 0.7352732434350603,
     "fpr": 0.10458715596330276,
     "tp": 518,
     "fp": 57
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 1.0,
     "real_skill_judge_calls_share": 1.0,
     "real_skill_flag_rate": 0.076
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.5877,
      0.6534
     ],
     "fpr": [
      0.0816,
      0.1331
     ]
    }
   },
   {
    "id": "screen-new-prompt_injection-low-confidence",
    "label": "OpenJev prompt_injection screen, then judge, new prompt, low-confidence findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.5419664268585132,
     "precision": 0.9658119658119658,
     "f1": 0.6943164362519202,
     "fpr": 0.029357798165137616,
     "tp": 452,
     "fp": 16
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5576504713560552,
     "real_skill_judge_calls_share": 0.0605,
     "real_skill_flag_rate": 0.0235
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.508,
      0.5755
     ],
     "fpr": [
      0.0182,
      0.0472
     ]
    }
   },
   {
    "id": "screen-new-logistic-low-confidence",
    "label": "OpenJev logistic screen (8 probes), then judge, new prompt, low-confidence findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.5683453237410072,
     "precision": 0.9813664596273292,
     "f1": 0.7198177676537586,
     "fpr": 0.01651376146788991,
     "tp": 474,
     "fp": 9
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5598259608411893,
     "real_skill_judge_calls_share": 0.0245,
     "real_skill_flag_rate": 0.0135
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.5345,
      0.6016
     ],
     "fpr": [
      0.0087,
      0.0311
     ]
    }
   },
   {
    "id": "screen-new-none-contextual",
    "label": "Judge alone, new prompt, contextual findings capped at LOW",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.49280575539568344,
     "precision": 0.9470046082949308,
     "f1": 0.6482649842271293,
     "fpr": 0.04220183486238532,
     "tp": 411,
     "fp": 23
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 1.0,
     "real_skill_judge_calls_share": 1.0,
     "real_skill_flag_rate": 0.03
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.459,
      0.5267
     ],
     "fpr": [
      0.0283,
      0.0625
     ]
    }
   },
   {
    "id": "screen-new-prompt_injection-contextual",
    "label": "OpenJev prompt_injection screen, then judge, new prompt, contextual findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.4448441247002398,
     "precision": 0.9763157894736842,
     "f1": 0.6112026359143329,
     "fpr": 0.01651376146788991,
     "tp": 371,
     "fp": 9
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5264684554024656,
     "real_skill_judge_calls_share": 0.0505,
     "real_skill_flag_rate": 0.0175
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4114,
      0.4787
     ],
     "fpr": [
      0.0087,
      0.0311
     ]
    }
   },
   {
    "id": "screen-new-logistic-contextual",
    "label": "OpenJev logistic screen (8 probes), then judge, new prompt, contextual findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.43285371702637887,
     "precision": 0.9836512261580381,
     "f1": 0.601165695253955,
     "fpr": 0.011009174311926606,
     "tp": 361,
     "fp": 6
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.4873096446700508,
     "real_skill_judge_calls_share": 0.018,
     "real_skill_flag_rate": 0.008
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3996,
      0.4667
     ],
     "fpr": [
      0.0051,
      0.0238
     ]
    }
   },
   {
    "id": "screen-shipped-none-nocap",
    "label": "Judge alone, shipped prompt",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.513189448441247,
     "precision": 0.8294573643410853,
     "f1": 0.6340740740740741,
     "fpr": 0.1614678899082569,
     "tp": 428,
     "fp": 88
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 1.0,
     "real_skill_judge_calls_share": 1.0,
     "real_skill_flag_rate": 0.128
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4793,
      0.547
     ],
     "fpr": [
      0.133,
      0.1947
     ]
    }
   },
   {
    "id": "screen-shipped-prompt_injection-nocap",
    "label": "OpenJev prompt_injection screen, then judge, shipped prompt",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.47961630695443647,
     "precision": 0.9569377990430622,
     "f1": 0.6389776357827476,
     "fpr": 0.03302752293577982,
     "tp": 400,
     "fp": 18
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5823060188542422,
     "real_skill_judge_calls_share": 0.07,
     "real_skill_flag_rate": 0.026
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4459,
      0.5135
     ],
     "fpr": [
      0.021,
      0.0516
     ]
    }
   },
   {
    "id": "screen-shipped-logistic-nocap",
    "label": "OpenJev logistic screen (8 probes), then judge, shipped prompt",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.4880095923261391,
     "precision": 0.9736842105263158,
     "f1": 0.6501597444089458,
     "fpr": 0.02018348623853211,
     "tp": 407,
     "fp": 11
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.564902102973169,
     "real_skill_judge_calls_share": 0.0255,
     "real_skill_flag_rate": 0.013
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4542,
      0.5219
     ],
     "fpr": [
      0.0113,
      0.0358
     ]
    }
   },
   {
    "id": "screen-shipped-none-low-confidence",
    "label": "Judge alone, shipped prompt, low-confidence findings capped at LOW",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.39448441247002397,
     "precision": 0.8204488778054863,
     "f1": 0.5327935222672066,
     "fpr": 0.13211009174311927,
     "tp": 329,
     "fp": 72
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 1.0,
     "real_skill_judge_calls_share": 1.0,
     "real_skill_flag_rate": 0.092
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3619,
      0.4281
     ],
     "fpr": [
      0.1062,
      0.1631
     ]
    }
   },
   {
    "id": "screen-shipped-prompt_injection-low-confidence",
    "label": "OpenJev prompt_injection screen, then judge, shipped prompt, low-confidence findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.3764988009592326,
     "precision": 0.9573170731707317,
     "f1": 0.5404475043029259,
     "fpr": 0.025688073394495414,
     "tp": 314,
     "fp": 14
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.546047860768673,
     "real_skill_judge_calls_share": 0.0565,
     "real_skill_flag_rate": 0.0165
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3443,
      0.4099
     ],
     "fpr": [
      0.0154,
      0.0427
     ]
    }
   },
   {
    "id": "screen-shipped-logistic-low-confidence",
    "label": "OpenJev logistic screen (8 probes), then judge, shipped prompt, low-confidence findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.3669064748201439,
     "precision": 0.9776357827476039,
     "f1": 0.5335658238884046,
     "fpr": 0.012844036697247707,
     "tp": 306,
     "fp": 7
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.5163161711385061,
     "real_skill_judge_calls_share": 0.02,
     "real_skill_flag_rate": 0.008
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3349,
      0.4002
     ],
     "fpr": [
      0.0062,
      0.0263
     ]
    }
   },
   {
    "id": "screen-shipped-none-contextual",
    "label": "Judge alone, shipped prompt, contextual findings capped at LOW",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.3273381294964029,
     "precision": 0.9191919191919192,
     "f1": 0.4827586206896552,
     "fpr": 0.044036697247706424,
     "tp": 273,
     "fp": 24
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 1.0,
     "real_skill_judge_calls_share": 1.0,
     "real_skill_flag_rate": 0.0205
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.2963,
      0.3599
     ],
     "fpr": [
      0.0298,
      0.0647
     ]
    }
   },
   {
    "id": "screen-shipped-prompt_injection-contextual",
    "label": "OpenJev prompt_injection screen, then judge, shipped prompt, contextual findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.2997601918465228,
     "precision": 0.9689922480620154,
     "f1": 0.45787545787545797,
     "fpr": 0.014678899082568808,
     "tp": 250,
     "fp": 8
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.43002175489485134,
     "real_skill_judge_calls_share": 0.0305,
     "real_skill_flag_rate": 0.0055
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.2696,
      0.3317
     ],
     "fpr": [
      0.0075,
      0.0287
     ]
    }
   },
   {
    "id": "screen-shipped-logistic-contextual",
    "label": "OpenJev logistic screen (8 probes), then judge, shipped prompt, contextual findings capped at LOW",
    "group": "screen+judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1379,
    "metrics": {
     "recall": 0.27697841726618705,
     "precision": 0.9788135593220338,
     "f1": 0.43177570093457945,
     "fpr": 0.009174311926605505,
     "tp": 231,
     "fp": 5
    },
    "hero": false,
    "superseded": false,
    "variant": "jev_screen (n=1379 test records)",
    "cost": {
     "judge_calls_share": 0.39158810732414795,
     "real_skill_judge_calls_share": 0.012,
     "real_skill_flag_rate": 0.004
    },
    "source": "results/large-scale.json jev_screen.results, large-scale.html#3–#5",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.2477,
      0.3083
     ],
     "fpr": [
      0.0039,
      0.0213
     ]
    }
   },
   {
    "id": "judgeprompt-shipped-nocap",
    "label": "Judge alone, shipped prompt (judge_prompt run)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.5160905840286055,
     "precision": 0.8310940499040307,
     "f1": 0.636764705882353,
     "fpr": 0.1614678899082569
    },
    "hero": false,
    "superseded": false,
    "variant": "judge_prompt (n=1384): new prompt 65.9% recall / 75.8% F1 vs 65.7% / 75.6% in jev_screen (n=1379)",
    "cost": {
     "real_skill_flag_rate": 0.128
    },
    "source": "results/large-scale.json judge_prompt.rows, large-scale.html#15–#16",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4823,
      0.5498
     ],
     "fpr": [
      0.133,
      0.1947
     ]
    }
   },
   {
    "id": "judgeprompt-shipped-low-confidence",
    "label": "Judge alone, shipped prompt, low-confidence findings capped at LOW (judge_prompt run)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.39809296781883197,
     "precision": 0.8226600985221675,
     "f1": 0.5365461847389559,
     "fpr": 0.13211009174311927
    },
    "hero": false,
    "superseded": false,
    "variant": "judge_prompt (n=1384): new prompt 65.9% recall / 75.8% F1 vs 65.7% / 75.6% in jev_screen (n=1379)",
    "cost": {
     "real_skill_flag_rate": 0.092
    },
    "source": "results/large-scale.json judge_prompt.rows, large-scale.html#15–#16",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3655,
      0.4316
     ],
     "fpr": [
      0.1062,
      0.1631
     ]
    }
   },
   {
    "id": "judgeprompt-shipped-contextual",
    "label": "Judge alone, shipped prompt, contextual findings capped at LOW (judge_prompt run)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "shipped",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.33134684147794996,
     "precision": 0.9205298013245033,
     "f1": 0.4872918492550395,
     "fpr": 0.044036697247706424
    },
    "hero": false,
    "superseded": false,
    "variant": "judge_prompt (n=1384): new prompt 65.9% recall / 75.8% F1 vs 65.7% / 75.6% in jev_screen (n=1379)",
    "cost": {
     "real_skill_flag_rate": 0.0205
    },
    "source": "results/large-scale.json judge_prompt.rows, large-scale.html#15–#16",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3003,
      0.3639
     ],
     "fpr": [
      0.0298,
      0.0647
     ]
    }
   },
   {
    "id": "judgeprompt-new-nocap",
    "label": "Judge alone, new prompt (judge_prompt run)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.6591179976162098,
     "precision": 0.8904991948470209,
     "f1": 0.7575342465753425,
     "fpr": 0.12477064220183487
    },
    "hero": true,
    "superseded": false,
    "variant": "judge_prompt (n=1384): new prompt 65.9% recall / 75.8% F1 vs 65.7% / 75.6% in jev_screen (n=1379)",
    "cost": {
     "real_skill_flag_rate": 0.1045
    },
    "source": "results/large-scale.json judge_prompt.rows, large-scale.html#15–#16",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.6264,
      0.6904
     ],
     "fpr": [
      0.0996,
      0.1552
     ]
    }
   },
   {
    "id": "judgeprompt-new-low-confidence",
    "label": "Judge alone, new prompt, low-confidence findings capped at LOW (judge_prompt run)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.6233611442193087,
     "precision": 0.9017241379310345,
     "f1": 0.7371388301620859,
     "fpr": 0.10458715596330276
    },
    "hero": false,
    "superseded": false,
    "variant": "judge_prompt (n=1384): new prompt 65.9% recall / 75.8% F1 vs 65.7% / 75.6% in jev_screen (n=1379)",
    "cost": {
     "real_skill_flag_rate": 0.076
    },
    "source": "results/large-scale.json judge_prompt.rows, large-scale.html#15–#16",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.5901,
      0.6555
     ],
     "fpr": [
      0.0816,
      0.1331
     ]
    }
   },
   {
    "id": "judgeprompt-new-contextual",
    "label": "Judge alone, new prompt, contextual findings capped at LOW (judge_prompt run)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.4958283671036949,
     "precision": 0.9476082004555809,
     "f1": 0.6510172143974962,
     "fpr": 0.04220183486238532
    },
    "hero": false,
    "superseded": false,
    "variant": "judge_prompt (n=1384): new prompt 65.9% recall / 75.8% F1 vs 65.7% / 75.6% in jev_screen (n=1379)",
    "cost": {
     "real_skill_flag_rate": 0.03
    },
    "source": "results/large-scale.json judge_prompt.rows, large-scale.html#15–#16",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4621,
      0.5296
     ],
     "fpr": [
      0.0283,
      0.0625
     ]
    }
   },
   {
    "id": "decomposed-gemma-4-26b-3",
    "label": "Decomposed judge, 3 specialized passes, Gemma 4 26B (full split)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "specialized",
    "passes": 3,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.5685,
     "precision": 0.8548,
     "f1": 0.6829,
     "fpr": 0.1486
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#1",
    "notes": null
   },
   {
    "id": "decomposed-gemma-4-26b-5",
    "label": "Decomposed judge, 5 specialized passes, Gemma 4 26B (full split)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "specialized",
    "passes": 5,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.6091,
     "precision": 0.8517,
     "f1": 0.7102,
     "fpr": 0.1633
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": {
     "input_tokens_per_skill": 24703,
     "output_tokens_per_skill": 1548,
     "source": "improvements.html#6 (five passes, 120-record sample)"
    },
    "source": "improvements.html#1",
    "notes": null
   },
   {
    "id": "decomposed-claude-haiku-4.5-3",
    "label": "Decomposed judge, 3 specialized passes, Claude Haiku 4.5 (full split)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": null,
    "prompt": "specialized",
    "passes": 3,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.7664,
     "precision": 0.8689,
     "f1": 0.8144,
     "fpr": 0.178
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#2",
    "notes": null
   },
   {
    "id": "subsample-one-pass-shipped",
    "label": "One pass, shipped prompt (400-record subsample, Claude Haiku 4.5)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": null,
    "prompt": null,
    "passes": 1,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (400-record subsample: 250 malicious, 150 benign)",
    "n": 400,
    "metrics": {
     "recall": 0.624,
     "precision": 0.8571,
     "f1": 0.7222,
     "fpr": 0.1733
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#3",
    "notes": "Subsample: only differences between arms are meaningful."
   },
   {
    "id": "subsample-three-passes-same-prompt-control",
    "label": "Three passes, same prompt (control) (400-record subsample, Claude Haiku 4.5)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": null,
    "prompt": null,
    "passes": 3,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (400-record subsample: 250 malicious, 150 benign)",
    "n": 400,
    "metrics": {
     "recall": 0.74,
     "precision": 0.8525,
     "f1": 0.7923,
     "fpr": 0.2133
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#3",
    "notes": "Subsample: only differences between arms are meaningful."
   },
   {
    "id": "subsample-three-passes-specialized",
    "label": "Three passes, specialized prompts (400-record subsample, Claude Haiku 4.5)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": null,
    "prompt": null,
    "passes": 3,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (400-record subsample: 250 malicious, 150 benign)",
    "n": 400,
    "metrics": {
     "recall": 0.804,
     "precision": 0.859,
     "f1": 0.8306,
     "fpr": 0.22
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#3",
    "notes": "Subsample: only differences between arms are meaningful."
   },
   {
    "id": "bymodel-claude-haiku-4.5-before",
    "label": "Claude Haiku 4.5, one pass (improvements.html#9)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": null,
    "prompt": null,
    "passes": 1,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.458,
     "f1": 0.587,
     "fpr": 0.156
    },
    "hero": false,
    "superseded": false,
    "variant": "improvements.html#9 (rounded to 0.1%)",
    "cost": null,
    "source": "improvements.html#9",
    "notes": null
   },
   {
    "id": "bymodel-claude-haiku-4.5-after",
    "label": "Claude Haiku 4.5, three specialized passes (improvements.html#9)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "claude-haiku-4.5",
    "arm": null,
    "prompt": null,
    "passes": 3,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.766,
     "f1": 0.814,
     "fpr": 0.178
    },
    "hero": false,
    "superseded": false,
    "variant": "improvements.html#9 (rounded to 0.1%)",
    "cost": null,
    "source": "improvements.html#9",
    "notes": null
   },
   {
    "id": "bymodel-gemma-4-26b-before",
    "label": "Gemma 4 26B, one pass (improvements.html#9)",
    "group": "judge",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": 1,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.49,
     "f1": 0.612,
     "fpr": 0.176
    },
    "hero": false,
    "superseded": false,
    "variant": "improvements.html#9 (rounded to 0.1%)",
    "cost": null,
    "source": "improvements.html#9",
    "notes": null
   },
   {
    "id": "bymodel-gemma-4-26b-after",
    "label": "Gemma 4 26B, three specialized passes (improvements.html#9)",
    "group": "decomposed",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": 3,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.569,
     "f1": 0.683,
     "fpr": 0.149
    },
    "hero": false,
    "superseded": false,
    "variant": "improvements.html#9 (rounded to 0.1%)",
    "cost": null,
    "source": "improvements.html#9",
    "notes": null
   },
   {
    "id": "meta-judge-only",
    "label": "Judge only (strided 1-in-5 sample, Gemma 4 26B, meta-analyzer fixed)",
    "group": "meta",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (strided one-in-five sample, 274 records)",
    "n": 274,
    "metrics": {
     "recall": 0.491,
     "precision": 0.839,
     "f1": 0.619,
     "fpr": 0.15
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#5",
    "notes": null
   },
   {
    "id": "meta-judge-plus-meta",
    "label": "Judge plus meta-analyzer (strided 1-in-5 sample, Gemma 4 26B, meta-analyzer fixed)",
    "group": "meta",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint (strided one-in-five sample, 274 records)",
    "n": 274,
    "metrics": {
     "recall": 0.327,
     "precision": 0.771,
     "f1": 0.46,
     "fpr": 0.147
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "improvements.html#5",
    "notes": null
   },
   {
    "id": "system-one-single-question",
    "label": "System One model (OpenJev, single disposition question), any-intervention",
    "group": "system-one",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-source-disjoint",
    "n": 1336,
    "metrics": {
     "recall": 0.1673,
     "precision": 1.0,
     "f1": 0.2866,
     "fpr": 0.0
    },
    "hero": true,
    "superseded": false,
    "variant": null,
    "cost": {
     "usd_per_corpus": 0.1755,
     "seconds_per_skill": 0.044,
     "corpus_wall_clock_seconds": 42,
     "usd_total_quoted": 0.18,
     "source": "RESULTS.md cost table and stage table (dataset)"
    },
    "source": "cascade.html#3",
    "notes": "48 errors excluded. The shipped --system-one-endpoint tier is advisory only (index.html)."
   },
   {
    "id": "preset-balanced-rules-medium_plus",
    "label": "Preset balanced, rules only (review at MEDIUM+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07985697258641239,
     "fpr": 0.04220183486238532
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.02
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.0634,
      0.1002
     ],
     "fpr": [
      0.0283,
      0.0625
     ]
    }
   },
   {
    "id": "preset-balanced-rules-high_plus",
    "label": "Preset balanced, rules only (block at HIGH+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07747318235995232,
     "fpr": 0.04036697247706422
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.0185
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.0612,
      0.0975
     ],
     "fpr": [
      0.0268,
      0.0604
     ]
    }
   },
   {
    "id": "preset-balanced-judge-medium_plus",
    "label": "Preset balanced + judge (review at MEDIUM+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.6674612634088201,
     "fpr": 0.15412844036697249
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.1165
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.6349,
      0.6985
     ],
     "fpr": [
      0.1262,
      0.1869
     ]
    }
   },
   {
    "id": "preset-balanced-judge-high_plus",
    "label": "Preset balanced + judge (block at HIGH+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.33730631704410013,
     "fpr": 0.06422018348623854
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.038
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3061,
      0.37
     ],
     "fpr": [
      0.0465,
      0.088
     ]
    }
   },
   {
    "id": "preset-low-noise-rules-medium_plus",
    "label": "Preset low-noise, rules only (review at MEDIUM+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07866507747318235,
     "fpr": 0.04220183486238532
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.0165
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.0623,
      0.0989
     ],
     "fpr": [
      0.0283,
      0.0625
     ]
    }
   },
   {
    "id": "preset-low-noise-rules-high_plus",
    "label": "Preset low-noise, rules only (block at HIGH+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.07747318235995232,
     "fpr": 0.04036697247706422
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.0165
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.0612,
      0.0975
     ],
     "fpr": [
      0.0268,
      0.0604
     ]
    }
   },
   {
    "id": "preset-low-noise-judge-medium_plus",
    "label": "Preset low-noise + judge (review at MEDIUM+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.6317044100119189,
     "fpr": 0.13394495412844037
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.0865
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.5985,
      0.6637
     ],
     "fpr": [
      0.1079,
      0.1651
     ]
    }
   },
   {
    "id": "preset-low-noise-judge-high_plus",
    "label": "Preset low-noise + judge (block at HIGH+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.33730631704410013,
     "fpr": 0.06422018348623854
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.036
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3061,
      0.37
     ],
     "fpr": [
      0.0465,
      0.088
     ]
    }
   },
   {
    "id": "preset-quiet-rules-medium_plus",
    "label": "Preset quiet, rules only (review at MEDIUM+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.041716328963051254,
     "fpr": 0.03669724770642202
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.011
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.0301,
      0.0575
     ],
     "fpr": [
      0.0239,
      0.056
     ]
    }
   },
   {
    "id": "preset-quiet-rules-high_plus",
    "label": "Preset quiet, rules only (block at HIGH+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.03694874851013111,
     "fpr": 0.03302752293577982
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.01
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.0262,
      0.052
     ],
     "fpr": [
      0.021,
      0.0516
     ]
    }
   },
   {
    "id": "preset-quiet-judge-medium_plus",
    "label": "Preset quiet + judge (review at MEDIUM+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "medium_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.5029797377830751,
     "fpr": 0.07155963302752294
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.0385
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.4692,
      0.5367
     ],
     "fpr": [
      0.0528,
      0.0963
     ]
    }
   },
   {
    "id": "preset-quiet-judge-high_plus",
    "label": "Preset quiet + judge (block at HIGH+)",
    "group": "preset",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": "new",
    "passes": null,
    "lens": "high_plus",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.33134684147794996,
     "fpr": 0.05504587155963303
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": {
     "real_skill_flag_rate": 0.029
    },
    "source": "results/large-scale.json profiles, recommendations.html#2",
    "notes": null,
    "metrics_ci": {
     "recall": [
      0.3003,
      0.3639
     ],
     "fpr": [
      0.0388,
      0.0775
     ]
    }
   },
   {
    "id": "chain-rules_only-block_only",
    "label": "Chain rules_only (block-only lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "block_only",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.3125,
     "precision": 0.9542,
     "f1": 0.4708,
     "fpr": 0.015
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#1",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%"
   },
   {
    "id": "chain-judge_only-block_only",
    "label": "Chain judge_only (block-only lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "block_only",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.655,
     "precision": 0.9324,
     "f1": 0.7695,
     "fpr": 0.0475
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#1",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: judge 100%"
   },
   {
    "id": "chain-jev_only-block_only",
    "label": "Chain jev_only (block-only lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "block_only",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.34,
     "precision": 1.0,
     "f1": 0.5075,
     "fpr": 0.0
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#1",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: System One 99%"
   },
   {
    "id": "chain-rules_then_judge-block_only",
    "label": "Chain rules_then_judge (block-only lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "block_only",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.655,
     "precision": 0.9324,
     "f1": 0.7695,
     "fpr": 0.0475
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#1",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%; judge 84%"
   },
   {
    "id": "chain-rules_then_jev-block_only",
    "label": "Chain rules_then_jev (block-only lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "block_only",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.48,
     "precision": 0.9697,
     "f1": 0.6421,
     "fpr": 0.015
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#1",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%; System One 83%"
   },
   {
    "id": "chain-rules_then_jev_then_judge-block_only",
    "label": "Chain rules_then_jev_then_judge (block-only lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "block_only",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.6725,
     "precision": 0.934,
     "f1": 0.782,
     "fpr": 0.0475
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#1",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%; System One 83%; judge 75%"
   },
   {
    "id": "chain-rules_only-any_intervention",
    "label": "Chain rules_only (any-intervention lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": null,
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.33,
     "precision": 0.9429,
     "f1": 0.4889,
     "fpr": 0.02
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#2",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%"
   },
   {
    "id": "chain-judge_only-any_intervention",
    "label": "Chain judge_only (any-intervention lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.8175,
     "precision": 0.7861,
     "f1": 0.8015,
     "fpr": 0.2225
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#2",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: judge 100%"
   },
   {
    "id": "chain-jev_only-any_intervention",
    "label": "Chain jev_only (any-intervention lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.535,
     "precision": 1.0,
     "f1": 0.6971,
     "fpr": 0.0
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#2",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: System One 99%"
   },
   {
    "id": "chain-rules_then_judge-any_intervention",
    "label": "Chain rules_then_judge (any-intervention lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "gemma-4-26b",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.8175,
     "precision": 0.7861,
     "f1": 0.8015,
     "fpr": 0.2225
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#2",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%; judge 84%"
   },
   {
    "id": "chain-rules_then_jev-any_intervention",
    "label": "Chain rules_then_jev (any-intervention lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.64,
     "precision": 0.9697,
     "f1": 0.7711,
     "fpr": 0.02
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#2",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%; System One 83%"
   },
   {
    "id": "chain-rules_then_jev_then_judge-any_intervention",
    "label": "Chain rules_then_jev_then_judge (any-intervention lens, balanced subset)",
    "group": "system-one-chain",
    "tool": "skill-scanner",
    "model": "jev-1.13.0",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "any_intervention",
    "population": "msb-balanced-800 (400 malicious, 400 benign)",
    "n": 800,
    "metrics": {
     "recall": 0.84,
     "precision": 0.7906,
     "f1": 0.8145,
     "fpr": 0.2225
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#2",
    "notes": "Different population from the test split; shown only behind the corpus filter. Skills reaching each stage: rules 100%; System One 83%; judge 75%"
   },
   {
    "id": "prompt-guard-0.5",
    "label": "Llama Prompt Guard 2 (22M) as a pre-filter, threshold 0.5",
    "group": "prompt-guard",
    "tool": "llama-prompt-guard-2-22m",
    "model": "Llama-Prompt-Guard-2-22M",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "flag",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0,
     "precision": null,
     "fpr": 0.0037
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#6",
    "notes": "Max-window probability over 512-token windows; AUC 0.611.",
    "unavailable": {
     "precision": "—"
    }
   },
   {
    "id": "prompt-guard-0.8",
    "label": "Llama Prompt Guard 2 (22M) as a pre-filter, threshold 0.8",
    "group": "prompt-guard",
    "tool": "llama-prompt-guard-2-22m",
    "model": "Llama-Prompt-Guard-2-22M",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "flag",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0,
     "precision": null,
     "fpr": 0.0
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#6",
    "notes": "Max-window probability over 512-token windows; AUC 0.611.",
    "unavailable": {
     "precision": "—"
    }
   },
   {
    "id": "prompt-guard-0.9",
    "label": "Llama Prompt Guard 2 (22M) as a pre-filter, threshold 0.9",
    "group": "prompt-guard",
    "tool": "llama-prompt-guard-2-22m",
    "model": "Llama-Prompt-Guard-2-22M",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "flag",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0,
     "precision": null,
     "fpr": 0.0
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#6",
    "notes": "Max-window probability over 512-token windows; AUC 0.611.",
    "unavailable": {
     "precision": "—"
    }
   },
   {
    "id": "prompt-guard-0.99",
    "label": "Llama Prompt Guard 2 (22M) as a pre-filter, threshold 0.99",
    "group": "prompt-guard",
    "tool": "llama-prompt-guard-2-22m",
    "model": "Llama-Prompt-Guard-2-22M",
    "arm": null,
    "prompt": null,
    "passes": null,
    "lens": "flag",
    "population": "msb-source-disjoint",
    "n": 1384,
    "metrics": {
     "recall": 0.0,
     "precision": null,
     "fpr": 0.0
    },
    "hero": false,
    "superseded": false,
    "variant": null,
    "cost": null,
    "source": "cascade.html#6",
    "notes": "Max-window probability over 512-token windows; AUC 0.611.",
    "unavailable": {
     "precision": "—"
    }
   }
  ],
  "index_kpis": [
   {
    "label": "Real skills the rules flag at MEDIUM+, shipped to final",
    "value": [
     0.0418,
     0.0214
    ],
    "literal": "4.18% → 2.14%",
    "backed_by": "large-scale.html#10 MEDIUM+ row; results/large-scale.json full.deterministic"
   },
   {
    "label": "Rules false-positive rate on the held-out test split",
    "value": [
     0.0844,
     0.0422
    ],
    "literal": "8.44% → 4.22%",
    "backed_by": "large-scale.html#11 msb-source-disjoint row; full.labelled_ab"
   },
   {
    "label": "Judge F1 on the held-out test split, new prompt",
    "value": [
     0.637,
     0.758
    ],
    "literal": "63.7% → 75.8%",
    "backed_by": "large-scale.html#15; judge_prompt.rows"
   },
   {
    "label": "Of 2,000 sampled real skills, flagged by the judge behind the OpenJev screen; 3.1% reach the judge",
    "value": 0.0175,
    "literal": "1.75%",
    "reach_judge": 0.031,
    "backed_by": "large-scale.html#3 logistic row; jev_screen.results"
   },
   {
    "label": "F1 with rules alone",
    "value": 0.1374,
    "literal": "13.74%",
    "backed_by": "judged.html#1 core_only; deterministic.html#1"
   },
   {
    "label": "F1 with the judge added",
    "value": 0.3768,
    "literal": "37.68%",
    "backed_by": "judged.html#1 core_judge"
   },
   {
    "label": "Precision with the judge added",
    "value": 0.7849,
    "literal": "78.49%",
    "backed_by": "judged.html#1 core_judge"
   },
   {
    "label": "How much a judged run moves against itself",
    "value": 0.14,
    "literal": "14.00%",
    "backed_by": "experiments.html#13 noise floor"
   }
  ]
 }
}