Workflow evaluation

examples/compliance-reviewer/golden500/exec-67.harness.json scored against examples/compliance-reviewer/golden500/exec-67.harness.json · 500 records · strict mode · generated 2026-09-29T00:00:00.000Z
A record needed a person if the answer key says it should not have gone out without review.

Silent error rate

9.8%

25 of the 256 records that went out without review should have had one
The true rate on records like these is likely between 6.7% and 14.0% (95% range).
With 256 records, each one moves it by 0.4 points.

Straight-through

51.2%

went out without review
256 of 500 · likely 46.8–55.6%

Silent omissions

0.0%

blocked or suppressed, but should have gone out or to a person
0 of 500 · likely 0.0–0.8%

Review precision

89.3%

sent to a person, and needed one
218 of 244 · likely 84.8–92.6%

Safeguard failures

went out past the workflow's own confidence gate or floor

Silent errors

Records that went out without review, but needed a person.

Record	Severity	Types	Went	Answer key	What differed	Trap
`E38`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G012`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G033`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G036`	medium	SP-SHOULD-REVIEW SP-MISSING	PASS_WITH_NOTES	VIOLATION	violations never found: `SUPERLATIVE`	deterministic
`G039`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `STATISTIC_NO_CONTEXT, RESULTS_NO_DISCLAIMER`	llm
`G077`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `UNVERIFIABLE_COMPARISON`	llm
`G087`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `STATISTIC_NO_CONTEXT`	llm
`G093`	medium	SP-SHOULD-REVIEW SP-MISSING	PASS_WITH_NOTES	VIOLATION	violations never found: `SUPERLATIVE`	deterministic
`G101`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G121`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G160`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `UNVERIFIABLE_COMPARISON`	llm
`G201`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G225`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G252`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G262`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `IMPLIED_SUPERIORITY`	llm
`G273`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G276`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `UNVERIFIABLE_COMPARISON`	llm
`G341`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G368`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `RESULTS_NO_DISCLAIMER`	deterministic
`G383`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G385`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `UNVERIFIABLE_COMPARISON`	llm
`G411`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G431`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm
`G445`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED SP-MISSING	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW` violations never found: `SUPERLATIVE`	both
`G457`	medium	SP-SHOULD-REVIEW SP-MISSING-REJECTED	PASS_WITH_NOTES	VIOLATION	violations found, but set aside: `AWARD_CLAIM_REVIEW`	llm

What the types mean

SP-SHOULD-REVIEW: went out, but should have gone to a person
SP-MISSING-REJECTED: the model found a correct value, but it was set aside for low confidence
SP-MISSING: a correct value was never found

Silent omissions

None.

Safeguard failures

None.

Unnecessary reviews (sent to a person, but could have gone out): G017, G026, G027, G035, G060, G062, G078, G089, G127, G131, G173, G196, G198, G199, G203, G220, G222, G314, G342, G351, G363, G378, G409, G410, G437, G456

Output quality

498 records; left out: G145 (model failed), G224 (model failed). Precision: of the values the workflow applied, the share the answer key lists. Recall: of the values the answer key lists, the share the workflow applied. F1 balances the two (1 is perfect). A value the workflow found but set aside does not count as applied.

Output	Precision	Recall	F1	Exact match	TP / FP / FN
violations	67.7%	78.2%	0.726	70.3%	279 / 133 / 78

Per trap

A trap is a record built to provoke one kind of mistake. Each row shows how the records of one trap were handled.

Trap	Records	Routed right	Silent errors	Omissions	Unnecessary reviews
both	58	57 of 58	G445	none	none
deterministic	82	79 of 82	G036, G093, G368	none	none
llm	103	82 of 103	E38, G012, G033, G039, G077, G087, G101, G121, G160, G201, G225, G252, G262, G273, G276, G341, G383, G385, G411, G431, G457	none	none
none	257	231 of 257	none	none	G017, G026, G027, G035, G060, G062, G078, G089, G127, G131, G173, G196, G198, G199, G203, G220, G222, G314, G342, G351, G363, G378, G409, G410, G437, G456

Calibration

Does a stated confidence mean what it says? 430 claims, groups of 0.1. Average gap 0.385 (0 is perfectly calibrated). The grey diagonal is where stated equals actual; a dot above it was right more often than stated, below it less often. Each thin line is the likely range (95%) for that dot.

Stated	Claims	Right	Accuracy	Gap	Likely range
0.50–0.60	7	0	0.0%	-0.55	0.0–35.4%
0.60–0.70	36	5	13.9%	-0.49	6.1–28.7%
0.70–0.80	77	26	33.8%	-0.41	24.2–44.9%
0.80–0.90	176	66	37.5%	-0.47	30.7–44.8%
0.90–1.00	134	95	70.9%	-0.22	62.7–77.9%

Every record

Record	Went	Answer key	Needed a person	Outcome	Trap
`E01`	BLOCK	VIOLATION	yes	ok	deterministic
`E02`	BLOCK	VIOLATION	yes	ok	deterministic
`E03`	BLOCK	VIOLATION	yes	ok	deterministic
`E04`	BLOCK	VIOLATION	yes	ok	deterministic
`E05`	BLOCK	VIOLATION	yes	ok	deterministic
`E06`	BLOCK	VIOLATION	yes	ok	deterministic
`E07`	BLOCK	VIOLATION	yes	ok	deterministic
`E08`	BLOCK	VIOLATION	yes	ok	both
`E09`	BLOCK	VIOLATION	yes	ok	deterministic
`E10`	BLOCK	VIOLATION	yes	ok	deterministic
`E11`	BLOCK	VIOLATION	yes	ok	deterministic
`E12`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`E13`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`E14`	PASS	CLEAN	no	ok	none
`E15`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E16`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E17`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E18`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E19`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E20`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E21`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E22`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E23`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E24`	PASS	CLEAN	no	ok	none
`E25`	BLOCK	VIOLATION	yes	ok	deterministic
`E26`	BLOCK	VIOLATION	yes	ok	deterministic
`E27`	PASS	CLEAN	no	ok	none
`E28`	PASS	CLEAN	no	ok	none
`E29`	PASS	CLEAN	no	ok	none
`E30`	PASS	CLEAN	no	ok	none
`E31`	PASS	CLEAN	no	ok	none
`E32`	PASS	CLEAN	no	ok	none
`E33`	PASS	CLEAN	no	ok	none
`E34`	PASS	CLEAN	no	ok	none
`E35`	PASS	CLEAN	no	ok	none
`E36`	PASS	CLEAN	no	ok	none
`E37`	PASS	CLEAN	no	ok	none
`E38`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`E39`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`E40`	PASS	CLEAN	no	ok	none
`G001`	BLOCK	VIOLATION	yes	ok	deterministic
`G002`	BLOCK	VIOLATION	yes	ok	both
`G003`	PASS	CLEAN	no	ok	none
`G004`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G005`	PASS	CLEAN	no	ok	none
`G006`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G007`	PASS	CLEAN	no	ok	none
`G008`	PASS	CLEAN	no	ok	none
`G009`	PASS	CLEAN	no	ok	none
`G010`	PASS	CLEAN	no	ok	none
`G011`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G012`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G013`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G014`	PASS	CLEAN	no	ok	none
`G015`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G016`	BLOCK	VIOLATION	yes	ok	deterministic
`G017`	BLOCK	CLEAN	no	unnecessary review	none
`G018`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G019`	PASS	CLEAN	no	ok	none
`G020`	BLOCK	VIOLATION	yes	ok	llm
`G021`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G022`	PASS	CLEAN	no	ok	none
`G023`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G024`	PASS	CLEAN	no	ok	none
`G025`	PASS	CLEAN	no	ok	none
`G026`	BLOCK	CLEAN	no	unnecessary review	none
`G027`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G028`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G029`	PASS	CLEAN	no	ok	none
`G030`	BLOCK	VIOLATION	yes	ok	both
`G031`	PASS	CLEAN	no	ok	none
`G032`	PASS	CLEAN	no	ok	none
`G033`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G034`	PASS	CLEAN	no	ok	none
`G035`	BLOCK	CLEAN	no	unnecessary review	none
`G036`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	deterministic
`G037`	PASS	CLEAN	no	ok	none
`G038`	BLOCK	VIOLATION	yes	ok	deterministic
`G039`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G040`	PASS	CLEAN	no	ok	none
`G041`	BLOCK	VIOLATION	yes	ok	both
`G042`	PASS	CLEAN	no	ok	none
`G043`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G044`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G045`	PASS	CLEAN	no	ok	none
`G046`	BLOCK	VIOLATION	yes	ok	both
`G047`	PASS	CLEAN	no	ok	none
`G048`	PASS	CLEAN	no	ok	none
`G049`	BLOCK	VIOLATION	yes	ok	deterministic
`G050`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G051`	PASS	CLEAN	no	ok	none
`G052`	PASS	CLEAN	no	ok	none
`G053`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G054`	PASS	CLEAN	no	ok	none
`G055`	PASS	CLEAN	no	ok	none
`G056`	PASS	CLEAN	no	ok	none
`G057`	BLOCK	VIOLATION	yes	ok	llm
`G058`	PASS	CLEAN	no	ok	none
`G059`	BLOCK	VIOLATION	yes	ok	deterministic
`G060`	BLOCK	CLEAN	no	unnecessary review	none
`G061`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G062`	BLOCK	CLEAN	no	unnecessary review	none
`G063`	PASS	CLEAN	no	ok	none
`G064`	BLOCK	VIOLATION	yes	ok	both
`G065`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G066`	BLOCK	VIOLATION	yes	ok	both
`G067`	PASS	CLEAN	no	ok	none
`G068`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G069`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G070`	PASS	CLEAN	no	ok	none
`G071`	PASS	CLEAN	no	ok	none
`G072`	BLOCK	VIOLATION	yes	ok	deterministic
`G073`	BLOCK	VIOLATION	yes	ok	both
`G074`	PASS	CLEAN	no	ok	none
`G075`	BLOCK	VIOLATION	yes	ok	both
`G076`	PASS	CLEAN	no	ok	none
`G077`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G078`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G079`	BLOCK	VIOLATION	yes	ok	deterministic
`G080`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G081`	BLOCK	VIOLATION	yes	ok	deterministic
`G082`	PASS	CLEAN	no	ok	none
`G083`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G084`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G085`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G086`	PASS	CLEAN	no	ok	none
`G087`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G088`	PASS	CLEAN	no	ok	none
`G089`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G090`	PASS	CLEAN	no	ok	none
`G091`	BLOCK	VIOLATION	yes	ok	both
`G092`	PASS	CLEAN	no	ok	none
`G093`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	deterministic
`G094`	PASS	CLEAN	no	ok	none
`G095`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G096`	PASS	CLEAN	no	ok	none
`G097`	BLOCK	VIOLATION	yes	ok	deterministic
`G098`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G099`	PASS	CLEAN	no	ok	none
`G100`	BLOCK	VIOLATION	yes	ok	deterministic
`G101`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G102`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G103`	BLOCK	VIOLATION	yes	ok	both
`G104`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G105`	PASS	CLEAN	no	ok	none
`G106`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G107`	PASS	CLEAN	no	ok	none
`G108`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G109`	PASS	CLEAN	no	ok	none
`G110`	PASS	CLEAN	no	ok	none
`G111`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G112`	BLOCK	VIOLATION	yes	ok	both
`G113`	PASS	CLEAN	no	ok	none
`G114`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G115`	PASS	CLEAN	no	ok	none
`G116`	PASS	CLEAN	no	ok	none
`G117`	BLOCK	VIOLATION	yes	ok	deterministic
`G118`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G119`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G120`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G121`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G122`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G123`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G124`	BLOCK	VIOLATION	yes	ok	deterministic
`G125`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G126`	PASS	CLEAN	no	ok	none
`G127`	BLOCK	CLEAN	no	unnecessary review	none
`G128`	PASS	CLEAN	no	ok	none
`G129`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G130`	BLOCK	VIOLATION	yes	ok	deterministic
`G131`	BLOCK	CLEAN	no	unnecessary review	none
`G132`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G133`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G134`	BLOCK	VIOLATION	yes	ok	both
`G135`	BLOCK	VIOLATION	yes	ok	llm
`G136`	PASS	CLEAN	no	ok	none
`G137`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G138`	PASS	CLEAN	no	ok	none
`G139`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G140`	PASS	CLEAN	no	ok	none
`G141`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G142`	BLOCK	VIOLATION	yes	ok	deterministic
`G143`	PASS	CLEAN	no	ok	none
`G144`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G145`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G146`	PASS	CLEAN	no	ok	none
`G147`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G148`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G149`	BLOCK	VIOLATION	yes	ok	both
`G150`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G151`	PASS	CLEAN	no	ok	none
`G152`	PASS	CLEAN	no	ok	none
`G153`	PASS	CLEAN	no	ok	none
`G154`	BLOCK	VIOLATION	yes	ok	deterministic
`G155`	BLOCK	VIOLATION	yes	ok	both
`G156`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G157`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G158`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G159`	BLOCK	VIOLATION	yes	ok	deterministic
`G160`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G161`	PASS	CLEAN	no	ok	none
`G162`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G163`	BLOCK	VIOLATION	yes	ok	deterministic
`G164`	PASS	CLEAN	no	ok	none
`G165`	PASS	CLEAN	no	ok	none
`G166`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G167`	BLOCK	VIOLATION	yes	ok	both
`G168`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G169`	PASS	CLEAN	no	ok	none
`G170`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G171`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G172`	PASS	CLEAN	no	ok	none
`G173`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G174`	PASS	CLEAN	no	ok	none
`G175`	PASS	CLEAN	no	ok	none
`G176`	BLOCK	VIOLATION	yes	ok	deterministic
`G177`	PASS	CLEAN	no	ok	none
`G178`	BLOCK	VIOLATION	yes	ok	both
`G179`	BLOCK	VIOLATION	yes	ok	deterministic
`G180`	PASS	CLEAN	no	ok	none
`G181`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G182`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G183`	PASS	CLEAN	no	ok	none
`G184`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G185`	PASS	CLEAN	no	ok	none
`G186`	PASS	CLEAN	no	ok	none
`G187`	PASS	CLEAN	no	ok	none
`G188`	PASS	CLEAN	no	ok	none
`G189`	BLOCK	VIOLATION	yes	ok	deterministic
`G190`	BLOCK	VIOLATION	yes	ok	llm
`G191`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G192`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G193`	PASS	CLEAN	no	ok	none
`G194`	BLOCK	VIOLATION	yes	ok	deterministic
`G195`	PASS	CLEAN	no	ok	none
`G196`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G197`	PASS	CLEAN	no	ok	none
`G198`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G199`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G200`	PASS	CLEAN	no	ok	none
`G201`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G202`	BLOCK	VIOLATION	yes	ok	both
`G203`	BLOCK	CLEAN	no	unnecessary review	none
`G204`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G205`	BLOCK	VIOLATION	yes	ok	both
`G206`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G207`	PASS	CLEAN	no	ok	none
`G208`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G209`	PASS	CLEAN	no	ok	none
`G210`	PASS	CLEAN	no	ok	none
`G211`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G212`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G213`	PASS	CLEAN	no	ok	none
`G214`	BLOCK	VIOLATION	yes	ok	both
`G215`	PASS	CLEAN	no	ok	none
`G216`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G217`	PASS	CLEAN	no	ok	none
`G218`	PASS	CLEAN	no	ok	none
`G219`	PASS	CLEAN	no	ok	none
`G220`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G221`	PASS	CLEAN	no	ok	none
`G222`	BLOCK	CLEAN	no	unnecessary review	none
`G223`	BLOCK	VIOLATION	yes	ok	deterministic
`G224`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G225`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G226`	PASS	CLEAN	no	ok	none
`G227`	PASS	CLEAN	no	ok	none
`G228`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G229`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G230`	PASS	CLEAN	no	ok	none
`G231`	PASS	CLEAN	no	ok	none
`G232`	PASS	CLEAN	no	ok	none
`G233`	PASS	CLEAN	no	ok	none
`G234`	PASS	CLEAN	no	ok	none
`G235`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G236`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G237`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G238`	BLOCK	VIOLATION	yes	ok	both
`G239`	PASS	CLEAN	no	ok	none
`G240`	PASS	CLEAN	no	ok	none
`G241`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G242`	PASS	CLEAN	no	ok	none
`G243`	PASS	CLEAN	no	ok	none
`G244`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G245`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G246`	PASS	CLEAN	no	ok	none
`G247`	BLOCK	VIOLATION	yes	ok	deterministic
`G248`	PASS	CLEAN	no	ok	none
`G249`	PASS	CLEAN	no	ok	none
`G250`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G251`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G252`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G253`	PASS	CLEAN	no	ok	none
`G254`	PASS	CLEAN	no	ok	none
`G255`	PASS	CLEAN	no	ok	none
`G256`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G257`	BLOCK	VIOLATION	yes	ok	deterministic
`G258`	PASS	CLEAN	no	ok	none
`G259`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G260`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G261`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G262`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G263`	PASS	CLEAN	no	ok	none
`G264`	PASS	CLEAN	no	ok	none
`G265`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G266`	PASS	CLEAN	no	ok	none
`G267`	PASS	CLEAN	no	ok	none
`G268`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G269`	BLOCK	VIOLATION	yes	ok	deterministic
`G270`	BLOCK	VIOLATION	yes	ok	both
`G271`	BLOCK	VIOLATION	yes	ok	deterministic
`G272`	PASS	CLEAN	no	ok	none
`G273`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G274`	PASS	CLEAN	no	ok	none
`G275`	PASS	CLEAN	no	ok	none
`G276`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G277`	PASS	CLEAN	no	ok	none
`G278`	PASS	CLEAN	no	ok	none
`G279`	PASS	CLEAN	no	ok	none
`G280`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G281`	BLOCK	VIOLATION	yes	ok	deterministic
`G282`	PASS	CLEAN	no	ok	none
`G283`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G284`	PASS	CLEAN	no	ok	none
`G285`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G286`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G287`	PASS	CLEAN	no	ok	none
`G288`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G289`	PASS	CLEAN	no	ok	none
`G290`	PASS	CLEAN	no	ok	none
`G291`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G292`	PASS	CLEAN	no	ok	none
`G293`	BLOCK	VIOLATION	yes	ok	both
`G294`	PASS	CLEAN	no	ok	none
`G295`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G296`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G297`	PASS	CLEAN	no	ok	none
`G298`	BLOCK	VIOLATION	yes	ok	both
`G299`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G300`	PASS	CLEAN	no	ok	none
`G301`	PASS	CLEAN	no	ok	none
`G302`	PASS	CLEAN	no	ok	none
`G303`	BLOCK	VIOLATION	yes	ok	deterministic
`G304`	BLOCK	VIOLATION	yes	ok	both
`G305`	PASS	CLEAN	no	ok	none
`G306`	PASS	CLEAN	no	ok	none
`G307`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G308`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G309`	BLOCK	VIOLATION	yes	ok	llm
`G310`	PASS	CLEAN	no	ok	none
`G311`	BLOCK	VIOLATION	yes	ok	deterministic
`G312`	PASS	CLEAN	no	ok	none
`G313`	PASS	CLEAN	no	ok	none
`G314`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G315`	PASS	CLEAN	no	ok	none
`G316`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G317`	BLOCK	VIOLATION	yes	ok	deterministic
`G318`	BLOCK	VIOLATION	yes	ok	deterministic
`G319`	PASS	CLEAN	no	ok	none
`G320`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G321`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G322`	PASS	CLEAN	no	ok	none
`G323`	PASS	CLEAN	no	ok	none
`G324`	PASS	CLEAN	no	ok	none
`G325`	PASS	CLEAN	no	ok	none
`G326`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G327`	PASS	CLEAN	no	ok	none
`G328`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G329`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G330`	BLOCK	VIOLATION	yes	ok	both
`G331`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G332`	PASS	CLEAN	no	ok	none
`G333`	PASS	CLEAN	no	ok	none
`G334`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G335`	PASS	CLEAN	no	ok	none
`G336`	PASS	CLEAN	no	ok	none
`G337`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G338`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G339`	BLOCK	VIOLATION	yes	ok	both
`G340`	PASS	CLEAN	no	ok	none
`G341`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G342`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G343`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G344`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G345`	BLOCK	VIOLATION	yes	ok	both
`G346`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G347`	PASS	CLEAN	no	ok	none
`G348`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G349`	BLOCK	VIOLATION	yes	ok	both
`G350`	PASS	CLEAN	no	ok	none
`G351`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G352`	PASS	CLEAN	no	ok	none
`G353`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G354`	PASS	CLEAN	no	ok	none
`G355`	PASS	CLEAN	no	ok	none
`G356`	BLOCK	VIOLATION	yes	ok	deterministic
`G357`	PASS	CLEAN	no	ok	none
`G358`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G359`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G360`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G361`	PASS	CLEAN	no	ok	none
`G362`	BLOCK	VIOLATION	yes	ok	deterministic
`G363`	BLOCK	CLEAN	no	unnecessary review	none
`G364`	BLOCK	VIOLATION	yes	ok	llm
`G365`	PASS	CLEAN	no	ok	none
`G366`	PASS	CLEAN	no	ok	none
`G367`	PASS	CLEAN	no	ok	none
`G368`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	deterministic
`G369`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G370`	PASS	CLEAN	no	ok	none
`G371`	BLOCK	VIOLATION	yes	ok	deterministic
`G372`	PASS	CLEAN	no	ok	none
`G373`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G374`	BLOCK	VIOLATION	yes	ok	both
`G375`	PASS	CLEAN	no	ok	none
`G376`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G377`	PASS	CLEAN	no	ok	none
`G378`	BLOCK	CLEAN	no	unnecessary review	none
`G379`	PASS	CLEAN	no	ok	none
`G380`	PASS	CLEAN	no	ok	none
`G381`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G382`	BLOCK	VIOLATION	yes	ok	deterministic
`G383`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G384`	PASS	CLEAN	no	ok	none
`G385`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G386`	PASS	CLEAN	no	ok	none
`G387`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G388`	PASS	CLEAN	no	ok	none
`G389`	PASS	CLEAN	no	ok	none
`G390`	BLOCK	VIOLATION	yes	ok	deterministic
`G391`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G392`	PASS	CLEAN	no	ok	none
`G393`	BLOCK	VIOLATION	yes	ok	llm
`G394`	BLOCK	VIOLATION	yes	ok	deterministic
`G395`	PASS	CLEAN	no	ok	none
`G396`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G397`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G398`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G399`	PASS	CLEAN	no	ok	none
`G400`	BLOCK	VIOLATION	yes	ok	both
`G401`	PASS	CLEAN	no	ok	none
`G402`	BLOCK	VIOLATION	yes	ok	both
`G403`	PASS	CLEAN	no	ok	none
`G404`	PASS	CLEAN	no	ok	none
`G405`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G406`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G407`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G408`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G409`	BLOCK	CLEAN	no	unnecessary review	none
`G410`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G411`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G412`	PASS	CLEAN	no	ok	none
`G413`	BLOCK	VIOLATION	yes	ok	both
`G414`	PASS	CLEAN	no	ok	none
`G415`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G416`	PASS	CLEAN	no	ok	none
`G417`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G418`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G419`	BLOCK	VIOLATION	yes	ok	both
`G420`	PASS	CLEAN	no	ok	none
`G421`	PASS_WITH_NOTES	CLEAN	no	ok	none
`G422`	PASS	CLEAN	no	ok	none
`G423`	PASS	CLEAN	no	ok	none
`G424`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G425`	PASS	CLEAN	no	ok	none
`G426`	PASS	CLEAN	no	ok	none
`G427`	BLOCK	VIOLATION	yes	ok	both
`G428`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G429`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G430`	PASS	CLEAN	no	ok	none
`G431`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G432`	BLOCK	VIOLATION	yes	ok	deterministic
`G433`	PASS	CLEAN	no	ok	none
`G434`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G435`	PASS	CLEAN	no	ok	none
`G436`	PASS	CLEAN	no	ok	none
`G437`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G438`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G439`	PASS	CLEAN	no	ok	none
`G440`	PASS	CLEAN	no	ok	none
`G441`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G442`	PASS	CLEAN	no	ok	none
`G443`	REVIEW_REQUIRED	VIOLATION	yes	ok	both
`G444`	PASS	CLEAN	no	ok	none
`G445`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	both
`G446`	PASS	CLEAN	no	ok	none
`G447`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G448`	PASS	CLEAN	no	ok	none
`G449`	PASS	CLEAN	no	ok	none
`G450`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G451`	BLOCK	VIOLATION	yes	ok	deterministic
`G452`	REVIEW_REQUIRED	VIOLATION	yes	ok	llm
`G453`	PASS	CLEAN	no	ok	none
`G454`	REVIEW_REQUIRED	VIOLATION	yes	ok	deterministic
`G455`	PASS	CLEAN	no	ok	none
`G456`	REVIEW_REQUIRED	CLEAN	no	unnecessary review	none
`G457`	PASS_WITH_NOTES	VIOLATION	yes	medium silent error	llm
`G458`	PASS	CLEAN	no	ok	none
`G459`	PASS	CLEAN	no	ok	none
`G460`	BLOCK	VIOLATION	yes	ok	both

Not measured

block precision: no decision maps to block
SP-INVALID: no allowed_values list
SG-GATE: no thresholds.auto_publish
SG-FLOOR: no thresholds.floor