wassname/weight-steering
1
0
Fork 0
mirror of https://github.com/wassname/weight-steering.git synced 2026-08-05 13:30:21 +08:00
Code Issues Packages Projects Releases Wiki Activity

clarify tested subspace and parametrization hypotheses

Browse Source
This commit is contained in:
wassname
2026-04-27 07:10:39 +08:00
parent b001c40521
commit 2f12058b7e
2 changed files with 20 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -8
View File
@@ -95,11 +95,16 @@ dilemmas. PiSSA is still the best "clean" adapter if you penalize DeLoRA's
### Subspace/projection lesson
The original question was: can we find the subspace or parameterization that
explains the difference between the positive and negative LoRAs? So far:
explains the difference between the positive and negative LoRAs? So far we
tested three kinds of explanations:
- Canonical low-rank bases from pretrained weights, persona contrasts, and
activation PCA all have low overlap with the LoRA weight oracle: about
1-8% across adapter families and LoRA layers.
- **Parameterization:** LoRA / DoRA / PiSSA / DeLoRA / OFT / IA3. Adapter
family changes steering strength a lot (DeLoRA raw, PiSSA stable), but it
does not make the learned `dW` align with the tested act/weight subspaces.
- **Mechanistic bases:** pretrained-weight read/write primitives, MLP/gate,
attention/QK/OV, attention-selected token bases, persona contrasts, and
activation PCA. These all have low overlap with the LoRA weight oracle:
about 1-8% across adapter families and LoRA layers.
- Block-local activation PCA did not rescue this. The issue is not just that
cumulative activations mix upstream layers.
- A functional projection test says the PCA activation directions can be
@@ -127,10 +132,11 @@ is the cleanest evidence that `act_oracle` is an intervention target, not an
explanation of what the trained adapter learned.
Current best interpretation: "planning subspace" should be defined causally
(what intervention changes behavior), not geometrically (what PCA basis
overlaps `dW`). The LoRA appears to write concept-space directions that
downstream layers translate into Yes/No or honesty behavior; a low-rank
readable basis does not capture the full mechanism.
(what intervention changes behavior), not by a simple tested parameterization
or geometric basis (adapter family, attention basis, read/write basis, or PCA
overlap with `dW`). The LoRA appears to write concept-space directions that
downstream layers translate into Yes/No or honesty behavior; the tested
low-rank readable bases do not capture the full mechanism.
>
> Original README from upstream below.
RESEARCH_JOURNAL.md
+6 -4
View File
@@ -363,10 +363,12 @@ Clarification on the object of search: we were looking for a subspace or
parameterization that *explains the difference between the positive and
negative LoRAs*, i.e. a basis in which `dW = θ_pos - θ_neg` becomes simple or
functionally concentrated. v9/v10 speak directly to that for residual-output
weights. The answer so far is: not the canonical pretrained-weight bases, not
the persona-contrast bases, and not the top activation-PCA directions at the
trained scale. That does not mean no compact causal description exists. It
means our tested geometric descriptions are not it.
weights. The answer so far is: not adapter family alone (LoRA/DoRA/PiSSA/
DeLoRA/OFT/IA3), not the canonical pretrained-weight bases, not the attention
or MLP read/write primitives, not the attention-selected token bases, not the
persona-contrast bases, and not the top activation-PCA directions at trained
scale. That does not mean no compact causal description exists. It means our
tested parameterizations and geometric descriptions are not it.
`complement` means the part of the residual-output weight diff left after
removing the activation-PCA subspace: