wassnameandClaude Opus 4.7 ada854c499 guided_rollout_multibool: switch to 12 single-slot KV-forks + assert full-attention ...

The chained-fill design (one suffix with all foundations, two passes for true/false)
hit a non-recoverable conv-state issue on hybrid linear-attention layers (Qwen3.5):
splitting prefix and suffix forwards via past_key_values silently produced
wrong logits (pmass dropping to 0.04, top token leaking to ' "' = 0.72).

Switched to 12 independent single-slot completions per prompt:
  for (frame, foundation) in {is_violation, is_ok} × foundations:
    cache scoring_prefix once, fork suffix `\n{"<frame>": {"<f>":`,
    read logits at the last token (predicting `true|false`).
  final[f] = 0.5 * (lr_violation[f] - lr_ok[f])

Framing flip cancels per-key prior bias the same way true/false fill did,
without the chained-slot causality that interacts badly with split forwards.

Added _assert_full_attention(): checks model.config.layer_types and fails
loudly on hybrid models. Verified parity vs flat forward on Qwen3-0.6B
(Δ ≤ 0.13 nats; signal of interest is ≫1 nat) and assert fires on Qwen3.5-0.8B.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-05 22:19:11 +08:00

..

tinymfv

guided_rollout_multibool: switch to 12 single-slot KV-forks + assert full-attention

2026-05-05 22:19:11 +08:00