mirror of
https://github.com/wassname/evil_MoE.git
synced 2026-09-12 16:30:37 +08:00
Add new scripts for AntiPaSTO and GRPO validation, including v_hack extraction, held-out validation, and smoke tests
This commit is contained in:
@@ -2,7 +2,57 @@
|
||||
|
||||
Append-only. New entries at the top, date-stamped. Never edit old entries.
|
||||
|
||||
# 2026-05-23
|
||||
# 2026-05-30
|
||||
|
||||
## Mechanism end-to-end verified on Qwen3.5-0.8B; H4 falsified at this scale
|
||||
|
||||
Closed the smoke loop: AntiPaSTO identity (bf16, max_abs_diff=0) -> v_hack
|
||||
extraction from 15 contrastive pairs -> held-out validation (frac>0=0.952,
|
||||
median cos=+0.363, n=186 modules) -> 10-step GRPO with subprocess-executed
|
||||
LeetCode rewards on vanilla and projected arms. Full writeup in
|
||||
[out/proof.md](../out/proof.md).
|
||||
|
||||
**Observation (mechanism)**: projected arm shows `cos_out < cos_in` every step,
|
||||
`frac_fired ≈ 0.51` averaged over 10 steps. Vanilla arm: `cos_out == cos_in`.
|
||||
The one-sided projection removes the v_hack-aligned component of the SVD-basis
|
||||
gradient when and only when alignment is positive. This is the core mechanical
|
||||
claim of the method and it is verified end-to-end.
|
||||
|
||||
**Observation (H4 sanity)**: both arms produce zero hack_rate and zero pass_rate
|
||||
on 30 LeetCode medium/hard problems, G=2, 10 steps. Inspection of generations
|
||||
shows Qwen3.5-0.8B emits format-only output that saturates the 0.25 format
|
||||
bonus but never attempts code or hack patterns. Per [spec.md](../spec.md) §H4,
|
||||
this falls below the 30% hack-rate threshold and triggers the model-scaling
|
||||
fallback.
|
||||
|
||||
**Inference**: 0.8B is too small to exhibit the failure mode the method
|
||||
targets. The mechanism is sound; the test substrate is not. Wu & Tang's
|
||||
Rebound paper used Qwen2.5-Coder-7B and observed ~50% baseline hack rate;
|
||||
Ariahw's benchmark assumes ≥4B class models. Mechanism + scale are
|
||||
separable concerns and the smaller scope of this session was mechanism.
|
||||
|
||||
**Caveats / what's untested**:
|
||||
|
||||
- β=0 (no ref-model KL) to fit 24 GB. Rebound used β=0.04. KL-free GRPO can
|
||||
diverge faster; not a fair comparison to Rebound at this scale.
|
||||
- Only 10 steps. Reward-hacking emerges around step 50–200 in Rebound figs.
|
||||
- 186 target modules, m=8 SVD rank. Larger models scale this to ~400+ modules.
|
||||
- `frac_fired ≈ 0.5` is consistent with random gradient direction wrt v_hack
|
||||
at init; we expect it to rise as training induces hack-aligned grads. Need
|
||||
longer runs to see this.
|
||||
|
||||
**Next (queued in [justfile](../justfile), pending ≥80 GB GPU)**:
|
||||
|
||||
1. `queue-vanilla`: Qwen2.5-Coder-7B baseline GRPO on full LeetCode set, 200
|
||||
steps, 3 seeds, β=0.04, G=4. Expected hack_rate at convergence: 40–60%
|
||||
(Rebound table 2).
|
||||
2. `queue-projected-m16`: same config + per-module v_hack projection at m=16.
|
||||
3. `queue-rebound`: H3 baseline arm — Wu-Tang advantage modification.
|
||||
|
||||
Confidence in method post-mechanism-verification: ~65% (was ~60%). The bump is
|
||||
small because mechanism-works was already high-prior; the real evidence is the
|
||||
7B run.
|
||||
|
||||
|
||||
## Project init
|
||||
|
||||
|
||||
Reference in New Issue
Block a user