wassname c4797cd732 expose pmass_format as aggregate signal ...

ForcedChoiceResult now carries pmass_format (sum prob mass on the K
foundation answer tokens at the JSON answer slot, averaged across fwd
and rev framings). eval.py aggregates it as mean_pmass_format in both
the headline return dict and the info subdict, and propagates per-row
for sweep/audit consumers.

Direct coherence canary: drops when steering pushes the model toward
non-foundation tokens (gibberish, refusal, format collapse). Independent
of which foundation is picked — complementary to top1_acc (label-
agreement; intentional target shift) and mean_nll_prompt (teacher-forced
prompt nll; falls under steering even when generations break).

Surfacing this lets downstream callers (weight-steering-lite walkback,
report dashboards) gate on actual coherence rather than misusing top1
as a budget.

2026-05-18 11:26:00 +00:00

..

__init__.py

refactor

2026-05-08 15:15:14 +08:00

data.py

rename

2026-05-08 15:30:06 +08:00

eval.py

expose pmass_format as aggregate signal

2026-05-18 11:26:00 +00:00

guided.py

expose pmass_format as aggregate signal

2026-05-18 11:26:00 +00:00