wassname c792ad3e5f Add LoRA-XS variant: train only r×r core R between frozen SVD factors ...

Bałazy et al. 2024 (arxiv 2405.17604). A=diag(Sr)Vhr, B=Ur frozen from
top-r SVD of W (W left intact); only the r×r R is trained, init normal(0,1e-5)
so the adapter ~ identity at t=0. ~25k params at r=32 (24 down_proj targets).
justfile: alpha=r (scale=1) and lr=4e-3, matching the ref LLaMA math config.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>

2026-06-18 19:48:40 +08:00

..

_cost.py

benchmark sweep: rot(U/both) ablation, whitening conclusion, cost rows

2026-06-17 06:17:53 +08:00

cost_report.py

Collapse antipasto family to one variant: rot(V) becomes canonical antipasto

2026-06-17 21:05:51 +08:00

metamath_gsm8k_benchmark.py

Add LoRA-XS variant: train only r×r core R between frozen SVD factors

2026-06-18 19:48:40 +08:00