diff --git a/README.md b/README.md index 0c5036c..c2c6b54 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # 🍝 AntiPaSTO: Self-Supervised Steering of Moral Reasoning [![arXiv](https://img.shields.io/badge/arXiv-2601.07473-b31b1b.svg)](https://arxiv.org/abs/2601.07473) -[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) +[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE) **Anti-Pa**rallel **S**ubspace **T**raining for **O**rdered steering. @@ -9,6 +9,9 @@ Gradient-based steering in SVD transformation space, trained on internal representations without preference labels. Human input: two contrasting words ("honest" vs "dishonest"). Transfers out-of-distribution to moral dilemmas where prompting fails. + +![Bidirectional control](docs/img/fig_bidirectional_demo.svg) + ## Quick Start ```sh @@ -52,7 +55,7 @@ RLHF seasons the outputs but leaves the internals bland. AntiPaSTO marinates the - Incomplete contrast pairs (self-supervised, no labels to garnish) - Cayley rotations on V (the secret sauce—keeps everything orthogonal) - Projection loss + TV coherence + monotonicity constraints -- 800 synthetic pairs, ~1hr on A100 (low simmer) +- 800 synthetic pairs, ~1hr (low simmer) **What you get**: - Single adapter—flip α from +1 to -1 to reverse the flavor @@ -83,12 +86,12 @@ def loss(model, x_cho, x_rej): return L_proj + B_coh + B_mono ``` -![Adapter architecture](docs/img/apastoadapter_architecture.svg) - -![Bidirectional control](docs/img/fig_bidirectional_demo.svg) - ![Loss geometry](docs/img/loss.svg) + + + + ## Project Layout ``` @@ -108,7 +111,7 @@ outputs/adapters/ # trained models (ready to serve) ## Acknowledgments -Built on the shoulders of: +Built on the shoulders of other chefs: - [RepEng](https://github.com/vgel/repeng) — arithmetic steering that inspired this gradient-based approach - [PiSSA](https://github.com/GraphPKU/PiSSA) — SVD-based adapter initialization - [SSVD](https://arxiv.org/abs/2409.07268) — rotating V for domain generalization diff --git a/tests/__init__.py b/tests/__init__.py deleted file mode 100644 index e69de29..0000000