# 🍝 AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representations [![arXiv](https://img.shields.io/badge/arXiv-2601.07473-b31b1b.svg)](https://arxiv.org/abs/2601.07473) [![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE) [![BlogPost](https://img.shields.io/badge/BlogPost-Read%20More-blue.svg)](https://www.lesswrong.com/posts/nWiwv4GN8aYqpnZKE/antipasto-self-supervised-value-steering-for-debugging) **Anti-Pa**rallel **S**ubspace **T**raining for **O**rdered steering. *Serving up data-efficient inner alignment, one satisfying rotation at a time.* Accepted to the CoLoRAI workshop, ICML 2026. > Gradient-based honesty steering trained as an adapter on the model's own representations, not outputs. Human input: two contrasting words, no preference labels. **How it works:** Train a single adapter (~1 hour on Gemma-3-1B). At inference, dial the steering coefficient: +1 for more honest, -1 for less, 0 for baseline. One adapter, bidirectional control. **Why use it?** As models get more capable, eval awareness rises: models detect when they're being tested and adjust their behavior. You can't trust their outputs, their chain-of-thought, or their stated values at face value. You need a method that operates on internal representations rather than outputs, so it works even when the model is gaming the eval. AntiPaSTO steers what the model actually computes. On DailyDilemmas, it outperforms prompting by 6.9x and works where prompting triggers refusal. Applications: - *Combat eval awareness*: steer toward credulity and honesty so the model takes the eval at face value and gives honest answers. - *Find deeper moral preferences*: ask moral questions with and without honesty steering. Do stated values change? - *Swap the assistant axis*: find it and replace it with a philosopher-king or poet ![Bidirectional control](docs/img/fig_bidirectional_demo.svg) ## Paper-reported results The table is from [paper v5](https://arxiv.org/abs/2601.07473), trained on 800 honesty persona pairs and tested on [DailyDilemmas](https://arxiv.org/abs/2410.02683), an external benchmark of 1,360 moral dilemmas. It is not an expected result from the current default configuration. An [independent reproduction](https://github.com/Foomax/nobody-is-looking/tree/master/replication/experiments/antipasto-self-supervised-honesty-steering-via-a--wassname) did not recover the Gemma-3-1B headline with the shipped preset; it did recover the Gemma-3-270M result. | Method | Steer F1 | Tgt% | Wrong% | Arb% | Pmass | |:-------|---------:|-----:|-------:|-----:|------:| | AntiPaSTO | **31.2**±5.3 | 29.9 | 1.9 | 47.0 | 0.95 | | Eng. Prompt | 13.0 | | | | | | Prompting | 4.5 | 10.0 | 1.3 | 13.4 | 0.99 | | ActAdd | 0.0 | 0.0 | 0.0 | 0.0 | 0.99 | 6.9x the Steering F1 of prompting, and it wins on 5 of 6 tested value axes. ActAdd is activation addition; Eng. Prompt is an engineered prompt following [AxBench](https://arxiv.org/abs/2501.17148). Full tables, ablations, and cross-model results in the [paper](https://arxiv.org/abs/2601.07473). ## Quick Start ### Bake your own ```sh uv sync --all-groups uv run pytest tests/test_config_presets.py -q uv run pytest tests/test_train.py::test_train_rnd -v uv run python nbs/train.py gemma270m-24gb uv run python nbs/train.py paper-v5-gemma1b-24gb --seed 42 ``` `gemma270m-24gb` is the full 24GB run. An independent two-seed rerun reported 35.5 mean F1; paper v5 reports 38.7 for this model. `paper-v5-gemma1b-24gb` matches the Gemma-3-1B hyperparameters reported in paper v5. It is an attempt, not a verified reproduction of the paper's 31.2 F1 headline. `gemma1b-24gb` remains the current exploratory preset. Do not call `nbs/train.py` with no preset: its default selects Gemma-3-12B. No pretrained adapter is currently published for the old Hugging Face identifier in earlier README versions. ## The Recipe RLHF seasons the outputs but leaves the internals bland. AntiPaSTO marinates the model's hidden states directly, no preference labels required, just two contrasting words simmered into 800 synthetic pairs. ![Incomplete contrast pairs](docs/img/incomplete_contrast_pairs_v2.svg) **Ingredients**: - Incomplete contrast pairs (self-supervised, no labels to garnish) - Cayley rotations on V (the secret sauce, keeps everything orthogonal) - Projection loss + TV coherence + monotonicity constraints - 800 synthetic pairs, ~1hr (low simmer) **What you get**: - Single adapter: flip α from +1 to -1 to reverse the flavor - Train on honesty, transfers to 1,360 unseen moral dilemmas (9 value dimensions) - Beats prompting by 6.9x on small models; gradient optimization where arithmetic steering (CAA) gets F1=0 - Suppression bypass: steers when prompting triggers refusal or meta-commentary ## Architecture *The pasta machine: SVD decomposition + Cayley rotations* ```python # Adapter: rotate in SVD space def forward(h, alpha): R_v = cayley(theta_v, alpha) # coefficient-scaled rotation S_scaled = S + alpha * delta_S return h @ W_res.T + h @ V @ R_v @ diag(S_scaled) @ U.T # Loss: antiparallel separation + coherence + ordering def loss(model, x_cho, x_rej): delta_pos = model(x_cho, +1) - model(x_rej, +1) - d_ref delta_neg = model(x_cho, -1) - model(x_rej, -1) - d_ref L_proj = symlog(delta_pos @ delta_neg) # want < 0 (antiparallel) B_coh = tv_barrier(p_ref, p_pi, entropy) # TV trust region B_mono = hinge(Delta_neg < 0 < Delta_pos) # ordered control return L_proj + B_coh + B_mono ``` ![Loss geometry](docs/img/loss.svg) ## Project Layout ``` antipasto/ # the kitchen config.py # canonical recipe metrics.py # taste testing train/ # cooking instructions peft_utils/ # pasta machine internals docs/ # diagrams, plating notes nbs/ # experimental dishes outputs/adapters/ # trained models (ready to serve) ``` ## Status *Still simmering.* Full research history (experiments, ablations, burnt batches) available on request. I am working on v2 which - removes SVD for full lora (I found that changing the loss to prevent drift allows this) - reduces init variance - more expressive personas - larger models - better metric If you would like to collaborate, please reach out. ## Acknowledgments Built on the shoulders of other chefs: - [@Foomax](https://github.com/Foomax) -- independent replication testing - [CAA / RepEng](https://github.com/vgel/repeng) -- arithmetic steering that inspired this gradient-based approach - [PiSSA](https://github.com/GraphPKU/PiSSA) -- SVD-based adapter initialization - [SSVD](https://arxiv.org/abs/2409.07268) -- rotating V for domain generalization - [PEFT](https://github.com/huggingface/peft) -- the adapter ecosystem - [DailyDilemmas](https://github.com/chrischiu/dailydilemmas) -- the evaluation benchmark ## Citation ```bibtex @misc{clark2026antipasto, title = {AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representations}, author = {Clark, Michael J.}, year = {2026}, eprint = {2601.07473}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2601.07473} } ``` Nano banana's attempt to draw the loss landscape, I'm not sure if it helps understand the loss, but I like it