Add reference-impl URLs to variant docstrings + V2 external review

- Fetch canonical reference impls for offline review:
  * peft_{lora,hra,delora,ia3}_layer.py + peft_lora_{dora,variants}.py
  * orig_pissa_init.py (MuLabPKU/PiSSA)
  * orig_hra_layer.py (DaShenZi721/HRA)
  * orig_delora.py (ExplainableML/DeLoRA author fork)
- Add reference-impl URLs to all 6 variant docstrings
- Document HRA gate=0 dead-grad issue and DoRA detach-omission in their docstrings
- Re-run external review (codex) with refs available -> docs/audit/variants_review_v2.md
  Major NEW findings vs paper-only review:
    * DeLoRA: scalar W.norm() should be per-input-channel norm(dim=0)
    * HRA: PEFT uses symmetric repeated-column init (no dead grad), not zero gate
    * IA3: FFN targets need input-side gating, not output, our up_proj advice wrong
    * All LoRA-family: cfg.dropout silently ignored (no-op)
    * DeLoRA: wnorm should be persistent buffer, not Parameter
  HRA and DeLoRA upgraded to BUGGY (from Partial)

src/lora_lite/variants/delora.py

@@ -1,6 +1,6 @@
 """DeLoRA: column-normalised A, B, scaled by lambda * ||W||_F / r.
 
-Bini et al. 2025  https://arxiv.org/abs/2503.18225
+Bini et al. 2025 (ICLR'25)  https://arxiv.org/abs/2503.18225
 
 Paper Eq. 8:    W' = W + (lambda * ||W||_F / r) B Xi A
 where Xi_{i,i} = 1 / (||b_i|| ||a_i||) makes each rank-1 component unit-norm.
@@ -12,7 +12,22 @@ Identity at t=0: paper uses kaiming init for both A and B with `lambda` initiali
 to 0 (or small) so the effective delta starts near zero. We honour that:
 default lambda0 == 0 gives bit-identity; user can override via variant_kwargs.
 
+KNOWN GRADIENT ISSUE (flagged by external review 2026-04-26):
+  With lambda0=0 the *forward* is identity but `A,B` get zero gradient on step 0
+  (delta = lambda * ... -> d_output/d_A is proportional to lambda). Only
+  `lora_lambda` moves first step. With lambda0>0, A,B train but identity is broken.
+  Paper's true initialization (frozen-copy trick, see Eq. 9) achieves both;
+  we do NOT implement that here.
+
 The frozen ||W||_F factor is captured once at init() into a buffer `lora_wnorm`.
+
+Reference implementations (for review/cross-check):
+  - DeLoRA paper authors (ExplainableML/DeLoRA) -- their fork of peft:
+    https://github.com/ExplainableML/DeLoRA/blob/main/peft/src/peft/tuners/delora.py
+    (offline: docs/refs/orig_delora.py)
+  - peft DeLoRA (upstreamed):
+    https://github.com/huggingface/peft/blob/main/src/peft/tuners/delora/layer.py
+    (offline: docs/refs/peft_delora_layer.py)
 """
 import torch
 import torch.nn.functional as F

src/lora_lite/variants/dora.py

@@ -6,6 +6,17 @@ At t=0:  B=0 -> V=W -> y_new = (m_init / ||W||_c) (Wx + 0) = Wx   when m_init =
 
 Limitation: requires materializing the dense weight to compute ||V||_c. v1 supports
 plain nn.Linear only; bnb 4/8-bit layers raise loudly.
+
+DEVIATION (numerical):
+  - We differentiate through ||V||_c every forward. The paper's sec. 4.3 suggests
+    a 'cost-saving' variant that detaches ||V|| in backward (gradient only flows
+    through V); we do NOT do that. Real impact: slower step, slightly different
+    gradient direction. Faithful to the eq.5 forward, not the optimized one.
+
+Reference implementations (for review/cross-check):
+  - peft DoRA (separate file under lora/):
+    https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/dora.py
+    (offline: docs/refs/peft_lora_dora.py)
 """
 import torch
 import torch.nn.functional as F

src/lora_lite/variants/hra.py

@@ -13,10 +13,25 @@ Identity at t=0: `lora_gate` is initialized to 0 and gates each Householder
 vector, so the effective u_i starts at 0 -> H_i = I -> R = I -> y' = y.
 At training time the gate scales the active reflection direction.
 
+KNOWN GRADIENT ISSUE (flagged by external review 2026-04-26):
+  Forward is `x + gate * (Rx - x)`. With gate=0 at init, d_output/d_U is
+  proportional to gate, so on step 0 ONLY `lora_gate` receives gradient;
+  `lora_U` is dead. Once gate moves off zero, U starts learning. This deviates
+  from the paper, which has no such gate -- paper uses orthogonal init of U so
+  R != I from step 0. We trade paper-faithful init for identity-at-init.
+
 OMITTED: paper also adds an orthogonality regularizer
     lambda * sum_i (u_i^T u_j)^2          (Eq. 6 / Sec. 3.3)
 which is a loss term, not a forward-pass change. Add it in your training loop if
 you want the regularized HRA variant.
+
+Reference implementations (for review/cross-check):
+  - HRA paper authors (DaShenZi721/HRA), llama variant of OFT layer with HRA:
+    https://github.com/DaShenZi721/HRA/blob/master/llama/peft/oft/layer_GS_HRA.py
+    (offline: docs/refs/orig_hra_layer.py)
+  - peft HRA layer (cleaner, includes apply_GS toggle for orthogonalization):
+    https://github.com/huggingface/peft/blob/main/src/peft/tuners/hra/layer.py
+    (offline: docs/refs/peft_hra_layer.py)
 """
 import torch
 from einops import einsum

src/lora_lite/variants/ia3.py

@@ -16,6 +16,11 @@ DEVIATION FROM PAPER:
 
     `up_proj` is the closest stand-in for "FFN intermediate" in gated-MLP blocks
     (Llama uses gate * up; gating the up branch is the IA3-spirit choice).
+
+Reference implementations (for review/cross-check):
+  - peft IA3 layer (uses ia3_l elementwise scaling, fan_in_fan_out aware):
+    https://github.com/huggingface/peft/blob/main/src/peft/tuners/ia3/layer.py
+    (offline: docs/refs/peft_ia3_layer.py)
 """
 import torch
 from torch import nn

src/lora_lite/variants/lora.py

@@ -3,6 +3,11 @@
     h = W x + (alpha/r) B A x
 
 Identity at t=0 from B=0. Faithful to the paper.
+
+Reference implementations (for review/cross-check):
+  - peft Linear.update_layer + lora_A/B init, forward:
+    https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/layer.py
+    (see docs/refs/peft_lora_layer.py for offline copy)
 """
 from einops import einsum
 from torch import nn

src/lora_lite/variants/pissa.py

@@ -2,6 +2,21 @@
 
 Meng et al. 2024  https://arxiv.org/abs/2404.02948
 W_eff(t=0) = W_res + B@A = W (numerically; bf16 round-trip not bit-exact).
+
+DEVIATION FROM PAPER (documented):
+  - Paper sets adapter scale = 1 (no alpha/r factor); we keep LoRA's alpha/r
+    pipeline so callers must pass alpha=r to get paper-faithful identity.
+  - Saved adapter does NOT include W_res; load() recomputes PiSSA init on the
+    *same-seed base* before overwriting A/B. Reload is exact only on identical
+    base weights.
+
+Reference implementations (for review/cross-check):
+  - PiSSA original (NeurIPS'24 spotlight) init script (SVD on dequant W):
+    https://github.com/MuLabPKU/PiSSA/blob/main/utils/init_pissa.py
+    (offline: docs/refs/orig_pissa_init.py)
+  - peft PiSSA flavor (init_lora_weights='pissa') in:
+    https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/layer.py
+    (offline: docs/refs/peft_lora_layer.py, see pissa_init / loftq_init paths)
 """
 import torch
 from einops import einsum