wassname/lora-lite
1
0
Fork 0
mirror of https://github.com/wassname/lora-lite.git synced 2026-06-27 18:43:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
9 Commits 3 Branches 0 Tags
7eeaeed206dea1bfbee124cdec1ecdaafb6cf6ef
Commit Graph

9 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
wassname 7eeaeed206 Verify all variants on bnb 4bit/8bit; HRA paper-faithful rewrite 
- Test all 6 variants against bnb.Linear8bitLt + Linear4bit in smoke
- bnb-friendly (LoRA, IA3, HRA, DeLoRA): identity err <= 2.4e-4
- bnb-incompatible (PiSSA, DoRA): fail-loud TypeError as expected
- HRA: rewrite to paper-faithful input-side reflections (h <- (I-2vv^T)h),
  fixing previous broken output-side formulation
- IA3: bypass dtype upcast for bnb (params stay fp16/quantized)
- DeLoRA: explicit type check rejecting non-nn.Linear (incl. bnb)
- adapter: special-case bnb param assignment via .data
- Re-verified Qwen0.6B HRA probe: drop=20.7%, id_err=0, reload=0
 2026-04-26 18:08:06 +08:00
wassname 0d929f93b3 feat(hra): add Householder Reflection Adaptation, hook-only/bnb-friendly + Qwen proof 2026-04-26 17:58:56 +08:00
wassname 43e620176c docs: record DoRA + IA3 Qwen-0.6B proof results (tasks 80, 81) 2026-04-26 17:54:54 +08:00
wassname 2abf616be6 feat(dora): add weight-decomposed LoRA variant for fp layers 2026-04-26 17:53:33 +08:00
wassname 699fde31bf feat: ia3 variant, real bnb 4bit/8bit smoke, dev guide split, user-only readme 2026-04-26 17:49:17 +08:00
wassname f2d9021511 ci: add publishable check workflow 2026-04-26 17:09:47 +08:00
wassname 69bf5f4e44 test: prove adapter training paths 2026-04-26 17:00:39 +08:00
wassname 4db5cee5a9 init 2026-04-26 14:10:20 +08:00
wassname de97724b65 init 2026-04-26 14:10:18 +08:00