wassname/evil_MoE
1
0
Fork 0
mirror of https://github.com/wassname/evil_MoE.git synced 2026-06-27 16:45:42 +08:00
Code Issues Packages Projects Releases Wiki Activity

add routeV_absorb_all: 100% absorption, no vector (H2 extreme control)

Browse Source 
Route the whole gradient of every knob-on rollout into the quarantine; the
deployed knob learns only from the knob-off exploration floor. Direction-free
(v_grad extracted but never enters f -> routing is purely by generation mode).
Config flag + _step_absorb_f holder + filter branch (reuses act_vote per-rollout
machinery) + per-step is_ablated stash. just smoke-absorb passes (keep=0.25/
rout=0.75 = the floor/knob-on split). Queued s43 as job 29 (frac=0.25).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-09 11:56:12 +00:00
parent 028b8fff68
commit 0538dbf2f1
2 changed files with 50 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
justfile
+23
View File
@@ -68,6 +68,15 @@ smoke-routeV *ARGS:
--teacher-pool-dir=out/pools/teacher_pool --mix-ratio=0.5 \
--eval-ablate-every=10 --eval-n-prompts=2 {{ ARGS }}
# 100%-absorption control (NO vector): route every knob-on rollout fully into the
# quarantine, keep only the knob-off floor (rollout_ablate_frac) in the deployed knob.
# Direction-free -> the v_grad is extracted but inert. Needs frac>0 or the knob never updates.
smoke-absorb *ARGS:
BEARTYPE=1 {{ TRAIN }} smoke --intervention=routeV --routeV-absorb-all \
--rollout-ablate-frac=0.5 \
--teacher-pool-dir=out/pools/teacher_pool --mix-ratio=0.5 \
--eval-ablate-every=10 --eval-n-prompts=2 {{ ARGS }}
# Run smoke twice: first warms the v_hack cache (cache-miss path), second hits
# the cache (cache-hit path). Catches scope/save bugs that only manifest in one.
smoke-both:
@@ -136,6 +145,20 @@ fast-projected *ARGS:
fast-lora-routeV *ARGS:
{{ TRAIN }} fast --intervention=routeV --adapter=lora_frozen_b --lora-r=32 {{ ARGS }}
# H: ABSORB-ALL control (100% absorption, NO vector). Route the WHOLE gradient of every
# knob-on rollout into the quarantine; the deployed knob learns ONLY from the knob-off
# exploration floor (rollout_ablate_frac). v_grad is extracted (authored pairs) but inert
# -> routing is purely by generation mode, no direction. The extreme of H2: is the
# quarantine-as-sink + floor-only-deploy enough to suppress, with zero direction?
# resolve: deploy_hack ~ best/random-V -> absorption alone suffices (direction adds nothing);
# deploy_hack >> -> direction is load-bearing after all.
queue-absorb seed='43':
pueue add -w "$PWD" -o 24 \
-l "why: routeV ABSORB-ALL (100% absorption, NO vector, frac=0.25) s{{seed}}; resolve: deploy_hack ~ best/random-V -> mode-routing alone suppresses (H2 extreme); >> -> direction needed" \
-- {{ TRAIN }} fast --intervention=routeV --routeV-absorb-all --rollout-ablate-frac=0.25 \
--vhack-pairs-path=out/pairsets/pairs_authored.json \
--seed={{seed}} --out-tag=_dir8_routeV_absorb_s{{seed}}
# H: vGROUT directionality set -- 6 arms, ONE seed, single-mode run_tests, on the
# FIXED eval (paper test set, base solve ~0.1). Tests whether routeV's deploy-hack
# suppression needs the REAL hack direction. resolve: real-V (rollout & per-token)