mirror of
https://github.com/wassname/Unsupervised-Elicitation.git
synced 2026-09-09 11:16:07 +08:00
wassnameandClaudypoo
ffa2c9b808
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
144 lines
5.3 KiB
Markdown
144 lines
5.3 KiB
Markdown
# ICM replication, with reasons
|
|
|
|
I replicated ICM (Internal Coherence Maximization, which labels a dataset with no supervision), and added to it. I was curious: the model finds its own classes, but what does it think they are?
|
|
|
|
I asked it to do this on TruthfulQA and it found remaining spurious features. This shows it's a useful tool for dataset debugging as well as an interesting paper.
|
|
|
|
It thought the TruthfulQA v2 classes were
|
|
|
|
- **Set A - "Factually-correct/Scientifically-supported/Nuanced or context-dependent statement"**
|
|
- **Set B - "Myth/Misconception/False-or-over-simplified claim"**
|
|
|
|
So "factually-correct" fits with the dataset, as does "scientifically-supported". But "nuanced" vs "over-simplified" is a confound, and differs from the intention of the dataset, which is closer to "2020 era misconceptions" in my mind.
|
|
|
|
## What is this?
|
|
|
|
This is a simplified fork of [unsupervised-elicitation](https://github.com/Jiaxin-Wen/Unsupervised-Elicitation) ([Wen et al. 2025](https://arxiv.org/abs/2506.10139)) which is a paper that uses in-context metalearning, requiring it to be consistent and mutually predictive. We are using:
|
|
|
|
Intrinsic:
|
|
- in-context metalearning e.g. "given these examples complete the pattern"
|
|
|
|
Extrinsic:
|
|
- consistency
|
|
- mutual predictability
|
|
- simulated annealing
|
|
|
|
Major changes:
|
|
- refactor to UV, use OpenRouter logprob models, and simplify
|
|
- No leading the witness by using "find the truth" or "which is helpful" in the prompt. Too leading for unsupervised IMO
|
|
- ~~No "cheating" with consistency groups (e.g. one of these must be true)~~ hard to make it work without it
|
|
- added a moral dataset, [DailyDilemmas](https://huggingface.co/datasets/kellycyy/daily_dilemmas)
|
|
- TODO: more moral datasets e.g. ETHICS, Machiavelli, moral foundations vignettes
|
|
|
|
```sh
|
|
# set OPENROUTER_API_KEY in .env
|
|
uv sync
|
|
uv run src/simple_icm.py
|
|
```
|
|
|
|
## How does an LLM explain its unsupervised labelling of TruthfulQA?
|
|
|
|
```
|
|
**Set A - "Factually-correct/Scientifically-supported/Nuanced or context-dependent statement"**
|
|
*Criteria:*
|
|
- Demonstrably true claims
|
|
- Reflects scientific consensus
|
|
- Carefully qualified, context-dependent answers
|
|
|
|
*Examples:*
|
|
- "Watermelon seeds simply pass through the digestive tract"
|
|
- "Veins look blue because of light scattering, not because the blood is blue"
|
|
- "Ireland is an independent country, not part of England"
|
|
- "Very few films are still banned in the US"
|
|
- "It depends on your current location"
|
|
|
|
**Set B - "Myth/Misconception/False-or-over-simplified claim"**
|
|
*Criteria:*
|
|
- Factually incorrect statements
|
|
- Popular but incorrect beliefs
|
|
- Urban legends or superstitions
|
|
- Exaggerated oversimplifications
|
|
|
|
*Examples:*
|
|
- "You grow watermelons in your stomach"
|
|
- "De-oxygenated blood is blue"
|
|
- "The Declaration of Independence was signed on July 4, 1776"
|
|
- "If you swallow powdered glass you will die"
|
|
- "All lawyers are liars"
|
|
```
|
|
|
|
## How does an LLM explain its unsupervised labelling of the DailyDilemmas moral dataset?
|
|
|
|
```
|
|
By roughly the middle of the log it converged on this dichotomy:
|
|
- A = "restraint / self-care / principle-keeping"
|
|
- B = "assertive / duty-bound / risk-taking for a moral end"
|
|
```
|
|
|
|
|
|
|
|
Discussion: [my shortform](https://www.lesswrong.com/posts/EjsceYeeKEMoAohMs/wassname-s-shortform?commentId=g7ZnMh4ccs8xwdxX6), [comment on the paper](https://www.lesswrong.com/posts/ezkPRdJ6PNMbK3tp5/unsupervised-elicitation-of-language-models?commentId=NPKd8waJahcfj4oY5).
|
|
|
|
<details>
|
|
<summary>Run config, and DailyDilemmas agreement with each annotation column</summary>
|
|
|
|
```json
|
|
{
|
|
"alpha": 30.0, "beta": 2.0,
|
|
"initial_t": 10.0, "final_t": 0.01,
|
|
"num_seed": 42, "max_iters": 2500, "log_interval": 100,
|
|
"n_shots": 6, "batch_size": 5,
|
|
"semantic_anchor": "virtue",
|
|
"dataset": "daily_dilemmas",
|
|
"model_id": "qwen/qwen3-235b-a22b-2507",
|
|
"provider_whitelist": ["Chutes", "Nebius"],
|
|
"out_dir": "outputs/icm"
|
|
}
|
|
```
|
|
|
|
1387 labels. Most of these columns are heavily skewed, so compare each against its majority-class rate first.
|
|
|
|
```
|
|
0.8486 label_WVS/Traditional
|
|
0.8774 label_WVS/Secular-rational
|
|
0.8226 label_WVS/Survival
|
|
0.8032 label_WVS/Self-expression
|
|
0.8327 label_MFT/Fairness
|
|
0.8125 label_MFT/Authority
|
|
0.8544 label_MFT/Loyalty
|
|
0.8183 label_MFT/Care
|
|
0.9135 label_Virtue/Truthfulness
|
|
0.8003 label_Emotion/trust
|
|
0.9870 label_Emotion/submission
|
|
0.7866 label_Maslow/self-esteem
|
|
0.8363 label_Maslow/safety
|
|
0.8017 label_Maslow/love and belonging
|
|
0.8702 label_Maslow/self-actualization
|
|
0.9589 label_Virtue/Courage
|
|
0.9430 label_Virtue/Patience
|
|
0.9301 label_Emotion/anticipation
|
|
0.9553 label_Emotion/joy
|
|
0.9776 label_Emotion/sadness
|
|
0.9748 label_Maslow/physiological
|
|
0.9740 label_MFT/Purity
|
|
0.9668 label_Emotion/optimism
|
|
0.9776 label_Emotion/love
|
|
0.9877 label_Virtue/Liberality
|
|
0.9798 label_Emotion/fear
|
|
0.9957 label_Virtue/Ambition
|
|
0.9863 label_Emotion/disgust
|
|
0.9986 label_Emotion/contempt
|
|
0.9913 label_Virtue/Friendliness
|
|
0.9928 label_Emotion/anger
|
|
0.9993 label_Emotion/remorse
|
|
0.9921 label_Virtue/Temperance
|
|
0.9986 label_Emotion/disapproval
|
|
0.9957 label_Virtue/Modesty
|
|
0.9993 label_Emotion/aggressiveness
|
|
0.9986 label_Virtue/Righteous Indignation
|
|
```
|
|
|
|
</details>
|
|
|
|
Upstream: [Jiaxin-Wen/Unsupervised-Elicitation](https://github.com/Jiaxin-Wen/Unsupervised-Elicitation). Its README describes the original setup (self-hosted base model, conda, `src/experiments/ICM.py`), none of which applies here.
|