# Spec: routeA -- score activations, route gradients (2026-06-11)

New arm id `routeA` (per the rename-on-logic-change rule): the gate input changes from
gradients to activations, so routeV results stay comparable only to routeV.

## Evidence motivating the change

RESEARCH_JOURNAL.md 2026-06-11 (b) correction, (c), and (d) Table 1. On the A>0
contrast (reward-hack vs non-reward-hack among positively-reinforced rollouts, where
the advantage baseline is near chance), across three independent emergence windows:

| score | v3 | v4 | v5 | mean | min |
|---|---|---|---|---|---|
| act_cos | 0.869 | 0.749 | 0.752 | 0.790 | 0.749 |
| act_dot | 0.870 | 0.747 | 0.747 | 0.788 | 0.747 |
| resid_dot | 0.905 | 0.721 | 0.756 | 0.794 | 0.721 |
| resid_cos | 0.916 | 0.700 | 0.804 | 0.807 | 0.700 |
| grad_cos (current gate space) | 0.838 | 0.617 | 0.455 | 0.636 | 0.455 |
| grad_dot | 0.809 | 0.607 | 0.455 | 0.623 | 0.455 |

The gradient score does not replicate; the activation and residual scores do, and form
one statistical cluster (per-window SE ~ 0.07). The activation score needs no advantage
reconstruction, no gate backward pass, and no hooks outside the adapter modules, and it
has the best worst-window AUROC, so it is the default.

Variants tested and found null on these windows (all within ~0.03 of baseline min,
inside one SE; logs /tmp/claude-1000/superS_v1.log, act_dot_tstat.log, pinning_followup.log):

- t-stat extraction (v = mean(D)/se(D) per coordinate over the 8 behavior_ pairs,
  clamped to |t| <= 3): act 0.867/0.756/0.719 vs mean-extraction 0.869/0.749/0.752.
  With 8 pairs the per-coordinate std has ~25 percent relative error; revisit when the
  authored-pair set grows. Implemented behind `vact_tstat` (default off).
- Super-S-space (project the residual stream onto the pooled eigenbasis of the residual
  writers/readers, optionally whitened by the pooled spectrum, per
  wassname/steering-lite super_sspace; scripts/diag_pinning_superS.py): whitened
  variants top out at min-window 0.721 (= raw resid dot); the best unwhitened
  rotation+top-64 variant (reader basis) reaches 0.740, above raw resid cos (0.700)
  but below act (0.747-0.749), and it is the maximum of a ~50-variant grid, so
  post-hoc selection inflation applies. No variant beats the act default.
- SNR module weighting / top-quartile module pruning for act: moves AUROC <= 0.02.
- Per-layer resid: L24 alone >= the 3-layer concat on every window (0.925/0.712/0.814).

## What changes

1. **Extraction** (`extract_vhack_act` or a mode of the existing extractor): forward each
   authored pair side once, capture the deployed bottleneck activation `h = A[:r] @ x`
   per wrapped module, mean over completion tokens. `v_act[name] = unitnorm(mean over
   pairs of (act_hack - act_clean))`. No NLL, no backward, no c-probe. Reference
   implementation: `scripts/diag_pinning.py` (`ActTap`, `_v_from`).
2. **Gate**: capture the same pooled acts during the existing no-grad `logpi_old` forward
   (one hook; that forward already runs every step). Per-rollout score = concatenated-
   module `dot(act, v_act)` with v unit-norm per module. routeV's pass-1
   forward+backward is deleted; routeA costs approximately the vanilla arm.
3. **Score is dot.** Empirically act_dot == act_cos to ~0.002 on all three windows, so
   the choice is structural, not empirical: dot is a plain linear readout (one einsum, no
   per-rollout norm), and cos's per-rollout normalization makes the score sensitive to
   the norm of the pooled activation (a length/style factor) in the denominator.
   Boundedness no longer matters for threshold portability because thresholds are set on
   online-standardized scores (point 4), not raw values. Log cos as a diagnostic.
4. **Labels/pinning**: label-free, from online statistics of the live scores only
   (journal entry 2026-06-11 (d); reference implementation `_otsu3` in
   scripts/diag_pinning.py). The mean+k*sd rule was wrong because the live scores are a
   MIXTURE (hack share 35-43% in the emergence windows), so mean+2/3sd lands beyond
   every distribution and routes nothing. The replacement, validated on all three
   windows: (a) maintain a rolling buffer of recent per-rollout scores (e.g. last 256;
   one 24-rollout batch is too few); (b) standardize by the buffer mean/std (display
   and logging comparability; the partition itself is affine-invariant); (c) winsorize
   at the 1/99 percentiles, then a two-threshold Otsu search (maximize three-class
   between-class variance) gives (t_lo, t_hi) -> keep | absorb | rout. Winsorizing is
   load-bearing: without it a single outlier buys a zone (v4 grad_dot keep zone was one
   point; v5 act rout precision was 0.00 before, 0.50 after). Bimodality guard -- only
   open the rout zone when the between-class variance gain over a single threshold is
   material (else an all-clean early batch gets split and clean updates are routed);
   exact criterion to be fixed at implementation. The residual gap to a perfect router
   is the SCORE's AUROC, not the threshold: even the oracle single cut only reaches
   P ~ 0.5-0.65 at useful recall on v4/v5, so pure high-precision routing needs a
   better score or more authored pairs, not better pinning.
5. **Routing payload unchanged**: three-way masks (m, d) on block outputs, quarantine
   ablation at deployment, refresh of v_act every `vhack_refresh_every` steps (now
   forward-only, so cheap), placebo arm = Haar-random v_act with identical machinery.
6. **Pairs**: `#all-in-one/behavior_` (8 pairs), unchanged -- it was the best or tied
   vector on every window.

## What this removes

- The gate's second forward+backward per step (routeV's main cost).
- The advantage-reconstruction dependence of the gate score.
- The c-probe (`grad_probe`) machinery from the training path (keep it available for
  diagnostics; `scripts/diag_pinning.py` still uses it).
- G_hack / gradient-vector extraction from the training path.

## Open questions (do not block implementation)

- **Bottleneck vs residual stream**: RESOLVED ENOUGH (pueue #21-23, journal (d) Table
  1). resid and act are within ~1 SE of each other on every window; the random r=32
  bottleneck is NOT the limiter. Default the gate to the bottleneck act (act_dot): it
  has the best worst-window AUROC (0.747 vs resid_cos 0.700), lives entirely in gate
  space (no hooks on decoder layers), and needs no layer-depth choice. resid (L24
  alone, not the concat) stays the diagnostic alternative; it is adapter-independent
  and closest to a plain steering-vector probe, one hook-point change away.
- **Texture-shortcut risk**: the act score may detect exploit-token surface text. The
  held-out hack-mode test (train v_act on pairs covering some modes, measure routing of
  an absent mode) distinguishes generalization from memorized texture.
- Threshold rule v2 (pre-emergence frozen stats vs per-batch online).

## Implementation plan

APPROVED by wassname 2026-06-11 ("ok great do it"), with one amendment: the deletion in
step 4 covers not just train.py's routeV branch but the whole gradient-gate stack --
`extract_vhack_grad.py` and every train.py helper that exists only for it
(`_build_v_grad`, `route_band_edges`, `_pair_cos`, `_lora2r_gate_labels`, the pass-1
`autograd.grad` block, `grad_probe=True` wiring). The c-probe mechanism itself stays in
lora2r.py because scripts/diag_pinning.py uses it for diagnostics; training never
enables it. Clean as you go; audit with a grep for routeV/v_grad/route_band/grad_probe
across src/, justfile, and scripts/verify_* after.

Ordered; each step is one commit with its verify gate.

1. **Extraction** (`src/vgrout/extract_vhack_act.py`): `extract_v_act(model, wrappers,
   names, pairs, tok, device, tstat=False) -> dict[name, Tensor[r]]`. For each pair
   side one no-grad forward of prompt+completion; capture the deployed bottleneck per
   wrapped module with the ActTap hook pattern from scripts/diag_pinning.py
   (`F.linear(x, A[:r])` in a forward hook, [B, L, r], mean over completion tokens).
   `v[name] = unitnorm(mean over pairs of (h_hack - h_clean))`; `tstat=True` divides
   the mean by the standard error over pairs and clamps |t| <= 3 before the unitnorm.
   No NLL, no backward, no c-probe. Verify gate: `scripts/verify_v_act.py` checks the
   extractor reproduces the cached pair features in out/diag/pinning_feats.pt
   (`pair_feats[("act", side)]`) on the v3 checkpoint to fp tolerance.
2. **Gate in train.py**: routeA pins masks BEFORE the grad-carrying forward, like
   none/absorb, so the routeV second backward disappears and routeA costs ~ the
   vanilla arm. During the existing no-grad logpi_old forward (train.py ~line 877),
   the same ActTap hooks capture per-rollout pooled acts (pad-mask mean over
   completion tokens, batched). Score per rollout = sum over modules of
   `<h_m, v_m>` (act_dot; one einsum on the [B, M, r] stack). Log cos alongside.
3. **Pinning**: rolling deque of the last 256 live scores. Per step: append the
   batch, standardize by buffer mean/std (logged z), winsorize at 1/99 percentiles,
   `_otsu3` (lift from scripts/diag_pinning.py into the train module or a shared
   util) -> (t_lo, t_hi) -> keep (0,0) | absorb (1,0) | rout (1,1). Warmup: gate
   pinned absorb until the buffer holds >= 128 scores. Bimodality guard (proposal,
   to confirm at implementation): open the rout zone only when the high-class
   separation (mean of rout class minus mean of keep class) exceeds 1 buffer sd;
   otherwise collapse rout into absorb for that step.
4. **Arm wiring**: `intervention="routeA"` (rename-on-logic-change; routeV results
   stay comparable only to routeV). routeV and the whole gradient-gate stack are
   REMOVED in the same commit: src/vgrout/extract_vhack_grad.py, and train.py's
   `_build_v_grad`, `route_band_edges`, `_pair_cos`, `_lora2r_gate_labels`, the pass-1
   `autograd.grad` block, and `grad_probe=True` wiring. The c-probe mechanism stays in
   lora2r.py only because scripts/diag_pinning.py uses it for diagnostics; training
   never enables it. Placebo flag `routeA_random_v_seed` = Haar-random unit v_act per
   module, identical machinery. Refresh: reuse `vhack_refresh_every` (forward-only
   now, so cheap).
5. **Logging** (per step): gate AUROC on the A>0 contrast vs hack labels (diagnostic
   only; labels never feed routing), zone shares keep/absorb/rout, buffer mean/sd,
   (t_lo, t_hi) in z units, qmass. SHOULD lines per token-efficient-logging.
6. **Runs**: `just smoke` with routeA on the tiny model, then the fast preset pair:
   routeA real + routeA placebo + vanilla, seed 43, with the journal-(d) windows as
   the comparison reference.

## UAT

1. `just smoke` passes with the routeA pathway (act capture + gate labels on tiny model).
2. A routeA fast run logs per-step gate AUROC (A>0 contrast, students+cached teachers)
   >= 0.7 around first_hack, and deploy hack rate < vanilla at matched solve rate.
3. The Haar-placebo routeA arm does NOT match real v_act (directionality, not shrinkage).
