mirror of
https://github.com/wassname/evil_MoE.git
synced 2026-06-30 11:08:07 +08:00
wassname
997de37b26
Wassname flagged the dynamics curve wasn't comparable: route2 plotted its deploy eval (n=64, T=0.7, every 5 steps) while vanilla/erase plotted training rollouts (n=28, every step) -- route2 looked artificially smoother. (NOT a temperature gap: both gens are T=0.7; the "held-out greedy" header was a stale lie, now corrected.) train.py: ungate the periodic DEPLOY-eval to run for EVERY arm. route/route2 wrap it in ablate_quarantine (deploy = knob zeroed); vanilla/erase use nullcontext (deploy == trained model). Same estimator across arms. Cost: ~+40% amortized generation on the arms that newly get it (n=64 every 5 steps over ~32 train gens/step) -- n stays 64 to match the finished route2 n=3. plot_dynamics.py: plot hk_dep/slv_dep for ALL arms when present (drop the route-only guard; old logs fall back to training hack_s). Drop the cos row (it was for online-vs-offline erasure; not informative next to the rate row, and the cross-arm cos comparison was apples-to-oranges) -> single-row small multiples, "deployed rate". Title states deploy-eval n=64 T=0.7. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>