expand confound audit docs

This commit is contained in:
wassname
2026-06-13 14:43:03 +08:00
parent ae3fc096d7
commit 5b92bdf7a7
4 changed files with 71 additions and 13 deletions
+22 -1
View File
@@ -388,10 +388,12 @@ Low score can mean either no intended-axis movement or too much confounding. Rea
## Confounds Audited
The judge audits length, generic helpfulness, harmlessness/refusal, honesty/truthfulness, confidence, hedging, vagueness, warmth, enthusiasm, praise/flattery, sycophancy, formality, language shift, incoherence/repetition/rambling, persona echo, and generic off-axis helpfulness.
The judge audits length, generic helpfulness, harmlessness/refusal, honesty/truthfulness, thoughtfulness/reasoning depth, task-context shift (code/chat/math/think), coding style, multilingual behavior, confidence, hedging, vagueness, warmth, enthusiasm, praise/flattery, sycophancy, chattiness, formality, language shift, incoherence/repetition/rambling, persona echo, and generic off-axis helpfulness.
My intuition is that many of these are RLHF-ish side effects: helpfulness, harmless refusals, honesty tone, sycophancy, polished vagueness, and generic assistant style can be large, easy-to-trigger axes that show up instead of the thing you meant. - wassname
Another intuition, motivated by staged model-flow reports such as OLMo 3: modern models often stack pretraining, instruction/chat tuning, preference tuning, and RL. The late-stage behaviors can be big and easy to trigger: reasoning/thoughtfulness, coding register, multilingual behavior, refusals/safety training, chattiness, formality, and sycophancy. - wassname
## Tables
1. `main`: one row per measured template/persona-pair cell.
@@ -407,7 +409,26 @@ This library samples from or was shaped by:
- Assistant Axis: https://github.com/safety-research/assistant-axis
- weight-steering: https://github.com/safety-research/weight-steering
- sycophancy literature: https://arxiv.org/abs/2310.13548
- OLMo 3 report: https://arxiv.org/abs/2512.13961
- wassname/w2schar-mini: https://github.com/wassname/w2schar-mini
## Citation
```bibtex
@misc{{wassname_persona_steering_template_library_2026,
title = {{Persona Steering Template Library}},
author = {{Wassname}},
year = {{2026}},
url = {{https://github.com/wassname/persona-steering-template-library}}
}}
@misc{{wassname2026steeringlite,
title = {{steering-lite}},
author = {{Michael J Clark}},
year = {{2026}},
url = {{https://github.com/wassname/steering-lite}}
}}
```
"""
@@ -475,7 +475,12 @@ STYLE_DIM_DESCRIPTIONS = {
"helpfulness": "generic helpfulness or assistant helpful tone",
"harmlessness_refusal": "safety refusal, avoidance, or harmlessness framing",
"honesty_truthfulness": "truthfulness, correction, or epistemic honesty",
"thoughtfulness_reasoning": "deliberate reasoning, step-by-step thoughtfulness, or reflective depth",
"task_context_shift": "different task mode or domain, such as code, chat, math, or think-mode",
"coding_style": "programming/code-like structure, implementation detail, or software-engineering register",
"multilinguality": "non-English language use, translation-like behavior, or multilingual/code-switching",
"verbosity": "longer/more elaborated",
"chattiness": "chatty assistant tone, conversational filler, or over-engagement",
"confidence": "certainty/assertiveness",
"hedging": "caveats, uncertainty, maybe/likely language",
"vagueness": "generic, underspecified, or avoids concrete commitments",
@@ -496,6 +501,10 @@ OFF_AXIS_CONFOUNDS = (
"helpfulness",
"harmlessness/refusal",
"honesty/truthfulness",
"thoughtfulness/reasoning depth",
"task context shift, such as code/chat/math/think",
"coding ability or coding style",
"multilingual behavior",
"confidence",
"hedging",
"vagueness",
@@ -503,6 +512,7 @@ OFF_AXIS_CONFOUNDS = (
"enthusiasm",
"praise/flattery",
"sycophancy",
"chattiness",
"formality",
"language shift",
"incoherence/repetition/rambling",