evil_MoE

mirror of https://github.com/wassname/evil_MoE.git synced 2026-06-27 16:30:30 +08:00

Files

T

wassname 19687087b0 feat(#30,#39): simple online gate -- band from current batch, no window/cloud; lr 1e-4

Gate band (mean + k*std) now computed from THIS batch's pooled positions each step
instead of a sliding window. Refresh-proof by construction (live rollouts scored vs
the current v_grad), so the v_grad-refresh window flush is gone. Drops route_window
config + collections import. SmokeConfig forces routing (mid=-1,rout=0) since random
tiny data never separates -> quarantine would never train -> pathway assert would fail.

lr 3e-4 -> 1e-4: 3e-4 diverged at step ~27 (lp_s +18->+73, rew_s->0 after clean
emergence 7-24); 1e-4 is the normal LoRA range and emergence was already fast.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>

2026-06-11 06:04:28 +00:00

__init__.py

rename python package projected_grpo -> vgrout

2026-06-05 14:51:48 +08:00

data.py

fix: rotate the unhackable (gt_only) subset per step, not frozen per pid

2026-06-10 06:14:08 +00:00

derisk_loopholes.py

cleanup: consolidate stale loaders and pair scripts

2026-06-09 12:47:32 +00:00

eval.py

refactor: collapse to lora2r-only (none/routeV/absorb); delete erase/antipasto/lora_frozen_b paths