wassnameandClaude Opus 4.7 ce5d8c349d guided: hybrid natural+forced eval (architecture-independent, no KV slicing) ...

Phase 1: batched generate with min_new_tokens=max_new_tokens so cache is uniform
length across the batch (no early stop at </think>). Phase 2: single batched
forced-suffix forward over that cache. Per-sample classification picks
gen.scores at the natural answer position (case a), forced logits (case b
interrupted), or NaN (case c emitted </think> but no answer).

Drops _slice_pkv_one + per-sample fork. The slice helper used layer.keys /
layer.values which crashes on Qwen3.5/3.6 LinearAttentionLayer (gated-delta-net
recurrent state has no .keys/.values). Uniform-length batched cache sidesteps
the cache surface entirely.

Bumps transformers>=5.7 for the Qwen3.5/3.6 gated-delta-net cached-forward
bugfix (resolves to 5.9.0).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-23 06:54:54 +00:00

..

tinymfv

guided: hybrid natural+forced eval (architecture-independent, no KV slicing)

2026-05-23 06:54:54 +00:00