Commit Graph

3 Commits

Author SHA1 Message Date
wassname 09349894ce results: QLoRA bs=3 ga=2 + lam_round_pow=-0.5 extends movement to r6 (peak -0.37 vs -0.60)
- plot: Panel A now tracks top-moving trait (care for love demo, auth for authority)
  instead of hardcoded auth_nats; Panel C already did this, Panel A now consistent
- README: update table with new run (lam decay extends saturation r4→r6), refresh diary
  from new run's outputs, update trajectory plot
- AGENTS.md: correct gotchas -- tau<operating_KL is the key constraint (tau=2.0 not 4.0);
  QLoRA + bs=3 ga=2 is the right default for better heal gradient estimates

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-06-10 07:36:09 +08:00
wassname 2b884c2fb9 docs: QLoRA is net ~2x slower (gen-bound loop), keep mask-before-softmax heal fix
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-06-09 13:39:08 +08:00
wassname 940a3742c5 scaffold steer_heal: spec, repo infra, vendored deps
Setup per setup-repo conventions: uv + justfile + fast-dev-run on
wassname/qwen3-5lyr-tiny-random, package under src/steer_heal (config +
pipeline skeleton). Stages fail fast with NotImplementedError pointing at
the docs/vendor module to port from.

Design in spec.md: distil a steering-lite mean-diff teacher vector (iso-KL
dosed) into a conditioned LoRA, heal incoherency with a KL-rev-to-original
barrier, fold each round via w2schar gated bake, eval on tinymfv. Three
uncertainty gates (filter / heal / iterate) each with a UAT artifact.

Base model google/gemma-3-1b-it (RTX 3090, 24GB). Reference repos vendored
under docs/vendor (gitignored): steering-lite, isokl, tinymfv, w2schar-mini.
The lighter three are editable path deps; w2schar (py3.13 + flash-attn) is
reference-only, we copy its adapter/bake/plot modules.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-06-04 09:49:31 +08:00