mirror of
https://github.com/wassname/steer-heal-love.git
synced 2026-09-11 12:50:48 +08:00
results: rmse outlier-KL barrier holds coherence over the loop; README + log-incoherence plot
Headline (gemma-3-4b-it s42, care-over-authority): aggregating the kl_rev barrier by rmse over token positions (not the mean) holds coherence flat at 0.997 across all 8 rounds, where the mean aggregate collapses to 0.62 by r7 (token loops). Mean dilutes the few incoherent positions under the tau gate; rmse is outlier-sensitive and fires on them. Cost is depth (rmse run leashes to base, trait stays shallow); matched control still running. - plot.py: coherence panel -> log-incoherence (1-coh, log axis, down=coherent); map coherence axis matches; red steer kept on the over-pipeline panels only. - heal.py: fix kl_agg=p95 crash (torch.quantile rejects bf16 -> .float()). - run.py: persist per-round adapter gens (adapter_gen) for the outputs table. - config.py: coh_floor early-stop knob. - README: results table (mean vs rmse), trajectory figure, outputs-over-loop appendix (per-round completions as quotes); spec persona corrected to pos-neg. - docs/reviews: kl_agg review, pool saturation test, care-lens plan. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
+6
-1
@@ -16,7 +16,7 @@ from pathlib import Path
|
||||
import srsly
|
||||
import tyro
|
||||
|
||||
from steer_heal.plot import write_trajectory
|
||||
from steer_heal.plot import write_report, write_trajectory
|
||||
|
||||
|
||||
def main(run_dir: Path, base_auth: float | None = None,
|
||||
@@ -49,6 +49,11 @@ def main(run_dir: Path, base_auth: float | None = None,
|
||||
png = write_trajectory(run_dir, stages)
|
||||
print(f"re-rendered {png} from {len(stages)} stages ({len(healed)} rounds)", file=sys.stderr)
|
||||
|
||||
gen_rounds = sorted(by_stage("adapter_gen"), key=lambda e: e["round"])
|
||||
if gen_rounds: # runs from before adapter_gen was persisted (e.g. #107/#108) have none
|
||||
out = write_report(run_dir, gen_rounds)
|
||||
print(f"re-rendered {out} from {len(gen_rounds)} rounds of gens", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
tyro.cli(main)
|
||||
|
||||
Reference in New Issue
Block a user