 wassname
|
d0b4c52740
|
External review: per-variant audit + design notes
- Two acpx external reviews (codex + opencode):
* docs/audit/variants_review.md: per-variant paper-vs-impl audit
* docs/audit/design_review.md: peft EVA / baukit / antipasto3 vs lora-lite
* docs/audit/SUMMARY.md: aggregate verdicts + 3 risks + 5 follow-ups
- docs/refs/: peft_eva.py, peft_eva_finetuning.py, baukit_nethook.py,
antipasto3_svd_adapter.py for offline reference
Findings: LoRA clean; PiSSA/DoRA/IA3/HRA/DeLoRA have documented partial deviations.
Top risks: init/grad tradeoffs hidden by coarse tests; qwen probe lacks strict
identity tol; IA3 target placement untested.
|
2026-04-26 19:01:29 +08:00 |
|
|
wassname
|
7eeaeed206
|
Verify all variants on bnb 4bit/8bit; HRA paper-faithful rewrite
- Test all 6 variants against bnb.Linear8bitLt + Linear4bit in smoke
- bnb-friendly (LoRA, IA3, HRA, DeLoRA): identity err <= 2.4e-4
- bnb-incompatible (PiSSA, DoRA): fail-loud TypeError as expected
- HRA: rewrite to paper-faithful input-side reflections (h <- (I-2vv^T)h),
fixing previous broken output-side formulation
- IA3: bypass dtype upcast for bnb (params stay fp16/quantized)
- DeLoRA: explicit type check rejecting non-nn.Linear (incl. bnb)
- adapter: special-case bnb param assignment via .data
- Re-verified Qwen0.6B HRA probe: drop=20.7%, id_err=0, reload=0
|
2026-04-26 18:08:06 +08:00 |
|
|
wassname
|
0d929f93b3
|
feat(hra): add Householder Reflection Adaptation, hook-only/bnb-friendly + Qwen proof
|
2026-04-26 17:58:56 +08:00 |
|
|
wassname
|
43e620176c
|
docs: record DoRA + IA3 Qwen-0.6B proof results (tasks 80, 81)
|
2026-04-26 17:54:54 +08:00 |
|
|
wassname
|
2abf616be6
|
feat(dora): add weight-decomposed LoRA variant for fp layers
|
2026-04-26 17:53:33 +08:00 |
|
|
wassname
|
699fde31bf
|
feat: ia3 variant, real bnb 4bit/8bit smoke, dev guide split, user-only readme
|
2026-04-26 17:49:17 +08:00 |
|
|
wassname
|
f2d9021511
|
ci: add publishable check workflow
|
2026-04-26 17:09:47 +08:00 |
|
|
wassname
|
69bf5f4e44
|
test: prove adapter training paths
|
2026-04-26 17:00:39 +08:00 |
|
|
wassname
|
4db5cee5a9
|
init
|
2026-04-26 14:10:20 +08:00 |
|
|
wassname
|
de97724b65
|
init
|
2026-04-26 14:10:18 +08:00 |
|