mirror of
https://github.com/wassname/AntiPaSTO.git
synced 2026-09-09 11:12:52 +08:00
readmr
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# 🍝 AntiPaSTO: Self-Supervised Steering of Moral Reasoning
|
||||
|
||||
[](https://arxiv.org/abs/2601.07473)
|
||||
[](LICENSE)
|
||||
[](LICENSE)
|
||||
|
||||
**Anti-Pa**rallel **S**ubspace **T**raining for **O**rdered steering.
|
||||
|
||||
@@ -9,6 +9,9 @@
|
||||
|
||||
Gradient-based steering in SVD transformation space, trained on internal representations without preference labels. Human input: two contrasting words ("honest" vs "dishonest"). Transfers out-of-distribution to moral dilemmas where prompting fails.
|
||||
|
||||
|
||||

|
||||
|
||||
## Quick Start
|
||||
|
||||
```sh
|
||||
@@ -52,7 +55,7 @@ RLHF seasons the outputs but leaves the internals bland. AntiPaSTO marinates the
|
||||
- Incomplete contrast pairs (self-supervised, no labels to garnish)
|
||||
- Cayley rotations on V (the secret sauce—keeps everything orthogonal)
|
||||
- Projection loss + TV coherence + monotonicity constraints
|
||||
- 800 synthetic pairs, ~1hr on A100 (low simmer)
|
||||
- 800 synthetic pairs, ~1hr (low simmer)
|
||||
|
||||
**What you get**:
|
||||
- Single adapter—flip α from +1 to -1 to reverse the flavor
|
||||
@@ -83,12 +86,12 @@ def loss(model, x_cho, x_rej):
|
||||
|
||||
return L_proj + B_coh + B_mono
|
||||
```
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
|
||||
|
||||
<!--  -->
|
||||
|
||||
## Project Layout
|
||||
|
||||
```
|
||||
@@ -108,7 +111,7 @@ outputs/adapters/ # trained models (ready to serve)
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
Built on the shoulders of:
|
||||
Built on the shoulders of other chefs:
|
||||
- [RepEng](https://github.com/vgel/repeng) — arithmetic steering that inspired this gradient-based approach
|
||||
- [PiSSA](https://github.com/GraphPKU/PiSSA) — SVD-based adapter initialization
|
||||
- [SSVD](https://arxiv.org/abs/2409.07268) — rotating V for domain generalization
|
||||
|
||||
Reference in New Issue
Block a user