mirror of
https://github.com/wassname/evil_MoE.git
synced 2026-08-19 01:10:25 +08:00
log: reprint step-table header every 50 rows; related-work: Piggyback learned-mask critique
Header reprint fixes the variable-width misread trap (20+ unlabeled cols, gn adjacent to lr). Records the anticipated Piggyback 'why not learn the routing mask' critique (answer: no-cheat withholds the per-rollout label a learned mask needs) and LoRA rank-deficiency as mild support for the low-rank hack subspace. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
@@ -377,6 +377,14 @@ one-liners are in docs/grad\_routing/related\_work.md.}
|
||||
the subset from a given task label, whereas we \emph{remove} an unwanted
|
||||
capability and pick the subset from a gradient signal ($\cos$ to
|
||||
$v_{\text{hack}}$), with no per-rollout label.
|
||||
% Anticipated critique (Piggyback learns its mask end-to-end via a differentiable
|
||||
% real-valued threshold): why is our route gate a per-step calibrated cosine
|
||||
% threshold rather than a learned mask? Answer for the rebuttal: a learned mask
|
||||
% needs a per-rollout supervision signal (the task label Piggyback has); we
|
||||
% deliberately withhold that (no-cheat invariant), so the gate must come from the
|
||||
% unsupervised hack-vs-clean cos gap, not a trained parameter.
|
||||
% LoRA's rank-deficiency finding is mild external support for our low-rank hack
|
||||
% subspace (~10 pairs => rank-10).
|
||||
% \TODO{abstract-only twins to verify+place: GRIFT (gradient fingerprints,
|
||||
% arXiv:2604.16242); Spilling the Beans (OOD self-report, arXiv:2511.06626).}
|
||||
\end{itemize}
|
||||
|
||||
@@ -1554,7 +1554,10 @@ def main(cfg: Config) -> int:
|
||||
"sec": time.time() - t0,
|
||||
}
|
||||
rows.append(row)
|
||||
# Stream this step as a row (header was printed before the loop).
|
||||
# Stream this step as a row. Reprint the header every 50 rows so long runs
|
||||
# stay readable without scrolling back (20+ unlabeled columns, no per-row label).
|
||||
if step > 0 and step % 50 == 0:
|
||||
logger.info(step_logger.header())
|
||||
logger.info(step_logger.row(row))
|
||||
with rollout_log_path.open("a") as fh:
|
||||
for rec in step_rollouts:
|
||||
|
||||
Reference in New Issue
Block a user