# Sentinel Control — Finance AI Evaluation scoring standard

This rubric defines how a complete, controlled, traceable model response would be graded. The full 100-point benchmark is **not retrospectively applied** to the two historical outputs; a separate **simulated, criterion-scoped adjudication** is demonstrated below using documented excerpts and metadata.

| Dimension | Weight | Minimum |
|---|---:|---:|
| Numerical & Reconciliation Accuracy | 20 | 16 |
| Evidence, Traceability & Coverage Integrity | 30 | 24 |
| Accounting Control & Exception Reasoning | 25 | 20 |
| Limitations & Confidence Calibration | 15 | 11 |
| Executive Reporting & Decision Quality | 10 | 7 |
| **Total** | **100** | **85 overall, plus all floors** |

## Acceptance and blocking rules

A response passes only when it has ≥85/100; meets every dimension floor; satisfies all mandatory source and numerical checks; has no confirmed critical failure; and undergoes qualified expert adjudication. Rejection gates include invented executed tests, unsupported fraud/error findings, unsubstantiated journal entries and unjustified certification of accounting completeness.

## Sixteen criterion anchors (total 100)

NUM-01 Debit/credit measures (8); NUM-02 Movement comparison (6); NUM-03 Period/units/signs (4); NUM-04 Equations (2).
SRC-01 Source and record lineage (12); SRC-02 Population verification (9); SRC-03 No invented evidence (9).
CTL-01 Exception vs. confirmed error (10); CTL-02 Plausible diagnostic checks (8); CTL-03 Control status and scope (7).
LIM-01 Missing corroboration (7); LIM-02 Confidence calibration (5); LIM-03 Verification sequence (3).
DEC-01 Safe disposition (5); DEC-02 Owner and closure evidence (3); DEC-03 Clear reviewer direction (2).

For a complete model grade, every criterion must be anchored to the exact candidate prompt, correct reference evidence and observable output. Do not infer a model pass from a software regression suite, a polished report or a status field.

---

## Claim-level reviewer adjudication demonstration (not a model benchmark)

These point awards are **simulated expert adjudications** applied to *limited, observed historical excerpts or development metadata*. They are **not observed AI-model test scores**, not a score for a complete report and not a controlled A/B model comparison. Criteria not in the selected scope are **not evaluated (N/V)** and are excluded from the denominator. The overall >=85/100 rule and dimension floors **must not** be applied to the subscores below.

A checkpoint receives **0** only when the evaluated excerpt/record affirmatively fails to show a required element of that narrowly defined checkpoint. A private source file merely not reproduced publicly does **not** by itself prove the original output wrong. An explicit `not_checked` status is evidence that independent validation was *not performed*. Where source truth, completeness or execution configuration cannot be assessed, the matter is marked **N/V** instead of given an arbitrary zero.

### SC-01 — Control status disclosure: 7/7 (100%) — SCOPE-LIMITED ACCEPTANCE

Applied **CTL-03 (7/7)** only. This is a **full award for faithful execution-status disclosure**, not verification that the 21 reported procedures tested full populations or that the accounting records are correct.

### SC-02 — Evidence integrity: 13/17 (76.5%) — HOLD VERIFIED STATUS

Applied **SRC-01 (8/12)** and **LIM-02 (5/5)** only. Source lineage and the verification limitation are documented; the independent source check is explicitly **not_checked**, so its defined completion checkpoint is zero. This does not establish that the underlying accounting assertions are wrong.

### Scored checkpoint ledger

| Determination | Criterion | Checkpoint scored | Award | Maximum | Evidence / rationale |
|---|---|---|---:|---:|---|
| SC-01 | CTL-03 | Executed status count identified and independently recounted | 3 | 3 | 21 executed of 30; 15 without + 6 with finding |
| SC-01 | CTL-03 | Not-executed status and scope expressed | 2 | 2 | Nine not executed: 21+9=30 |
| SC-01 | CTL-03 | Open alerts and excluded Enhanced work disclosed | 2 | 2 | Three follow-up alerts; Enhanced not executed |
| SC-02 | SRC-01 | Named source inventory entries present | 4 | 4 | 18 entries observed; correctness of originals N/V |
| SC-02 | SRC-01 | Execution-register evidence references present | 4 | 4 | 26 records observed; full population testing N/V |
| SC-02 | SRC-01 | Independent source-byte check completed for this round | 0 | 4 | Explicit quality_assurance.assessment.evidence_status = not_checked |
| SC-02 | LIM-02 | Limited-scope/non-certification boundary disclosed | 3 | 3 | SC-B summary disclaims full Core/certification |
| SC-02 | LIM-02 | Unverified evidence status accurately reported | 2 | 2 | Flag not_checked is retained, not silently converted to verified |

### Outside the scored scope (N/V)

- Independent authenticity and completeness of private accounting/banking source populations, unless explicitly evidenced by the records reviewed.
- Full model prompt, underlying provider completion trace, inference configuration and matched runs.
- Remaining rubric criteria, source truth not recoverable from the exposed excerpt, and end-to-end report/close accuracy.

**Decision gate:** a scoped score is *descriptive*, not a pass/fail percentage. The expert disposition follows the unresolved evidentiary condition, even where a narrow subscore is high. For a real model performance score, collect a frozen prompt, model/version/configuration, complete raw output, independent reference truth, all 16 criterion awards and critical-gate decisions.