From 8f50ff502050dbbf27450f7ae69e2d66b8de0f02 Mon Sep 17 00:00:00 2001 From: wassname Date: Mon, 29 Jun 2026 06:30:11 +0800 Subject: [PATCH] Switch tables back to lists (narrow-screen friendly); add Featured section at top; trim process-leak intro Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- README.md | 95 ++++++++++++++++++++++++------------------------------- 1 file changed, 42 insertions(+), 53 deletions(-) diff --git a/README.md b/README.md index 6b2fdaf..dbc8ff9 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,20 @@ A curated list of datasets and benchmarks for evaluating the moral and ethical behaviour of language models: moral dilemmas, social norms, moral foundations, ethics judgements, honesty, value orientations, personas, and model organisms. -Every entry has a public URL. Reading the tables: +Bare links are HuggingFace datasets (`load_dataset(...)`); `gh`, `code`, and `paper` link elsewhere. `*` marks a recommended starting point. Each line ends with row count and provenance, where provenance is a rough quality signal: `human`, `AI` (LLM-generated), `mix`, or `derived`. -- Link type: a dataset link goes to HuggingFace (`load_dataset(...)`); entries marked `gh` link to a GitHub repo; `code` and `paper` are extra links. -- `*` marks a recommended starting point. -- Src (provenance, a rough quality signal): `human` = human-written or curated, `AI` = LLM-generated or synthetic, `mix` = both, `derived` = reprocessed from another dataset. -- N is the row count from the HuggingFace datasets-server, or the unit the authors use. +## Featured -Recommended starting points: [kellycyy/AIRiskDilemmas](https://huggingface.co/datasets/kellycyy/AIRiskDilemmas) (decisions), [wassname/tiny-mfv](https://huggingface.co/datasets/wassname/tiny-mfv) (foundations eval) with [wassname/moral_stories_foundations](https://huggingface.co/datasets/wassname/moral_stories_foundations) as its matched training set. +Good places to start, expanded in the sections below. + +- `*` [kellycyy/AIRiskDilemmas](https://huggingface.co/datasets/kellycyy/AIRiskDilemmas) - dilemmas facing a future AI; which values it prioritises under risk. +- `*` [wassname/tiny-mfv](https://huggingface.co/datasets/wassname/tiny-mfv) - fast forced-choice moral-foundations eval. +- `*` [wassname/moral_stories_foundations](https://huggingface.co/datasets/wassname/moral_stories_foundations) - matched training set for the tiny-mfv eval. +- `*` [wassname/genies_preferences](https://huggingface.co/datasets/wassname/genies_preferences) - overlooked 59-shift testbed for OOD generalisation. ## Contents +- [Featured](#featured) - [Moral dilemmas and decisions](#moral-dilemmas-and-decisions) - [Social norms and moral foundations](#social-norms-and-moral-foundations) - [Ethics judgements](#ethics-judgements) @@ -25,71 +28,57 @@ Recommended starting points: [kellycyy/AIRiskDilemmas](https://huggingface.co/da ## Moral dilemmas and decisions -| Dataset | N | Src | Paper | What it tests | -|---|---|---|---|---| -| `*` [kellycyy/AIRiskDilemmas](https://huggingface.co/datasets/kellycyy/AIRiskDilemmas) · [code](https://github.com/kellycyy/LitmusValues) | 6k eval rows (20.8k full) | AI | [2505.14633](https://arxiv.org/abs/2505.14633) | Dilemmas facing a future AI system; litmus-tests which values it prioritises under risk. | -| [kellycyy/daily_dilemmas](https://huggingface.co/datasets/kellycyy/daily_dilemmas) | 1,360 dilemmas | AI | [2410.02683](https://arxiv.org/abs/2410.02683) | Everyday value-conflict dilemmas, GPT-4 generated then validated against r/AITA ("Am I the Asshole", a Reddit forum where posters ask if they were in the wrong). | -| [wassname/daily_dilemmas-self](https://huggingface.co/datasets/wassname/daily_dilemmas-self) | 1,242 pairs | derived | [2410.02683](https://arxiv.org/abs/2410.02683) | The `party='You'` slice of daily_dilemmas, symmetrized into per-value labels. The author now prefers AIRiskDilemmas above. | -| [wassname/machiavelli](https://huggingface.co/datasets/wassname/machiavelli) · [code](https://github.com/wassname/machiavelli_as_ds) | 139,269 nodes | human | [2304.03279](https://arxiv.org/abs/2304.03279) | Power, deception, and harm choices in human-written choose-your-adventure games, reshaped for LLM scoring without fine-tuning. | -| [wassname/machiavelli_character_scenarios](https://huggingface.co/datasets/wassname/machiavelli_character_scenarios) | 566 prompts | derived | [2304.03279](https://arxiv.org/abs/2304.03279) | Roleplay decision prompts selected for spread on social/moral labels (fairness, deception, manipulation, promises, spying). | +- `*` [kellycyy/AIRiskDilemmas](https://huggingface.co/datasets/kellycyy/AIRiskDilemmas) - dilemmas facing a future AI system; litmus-tests which values it prioritises under risk. [paper](https://arxiv.org/abs/2505.14633), [code](https://github.com/kellycyy/LitmusValues). *6k eval rows (20.8k full), AI.* +- [kellycyy/daily_dilemmas](https://huggingface.co/datasets/kellycyy/daily_dilemmas) - everyday value-conflict dilemmas, GPT-4 generated then validated against r/AITA (Reddit's "Am I the Asshole", where posters ask if they were in the wrong). [paper](https://arxiv.org/abs/2410.02683). *1,360 dilemmas, AI.* +- [wassname/daily_dilemmas-self](https://huggingface.co/datasets/wassname/daily_dilemmas-self) - the `party='You'` slice of daily_dilemmas, symmetrized into per-value labels. The author now prefers AIRiskDilemmas. [paper](https://arxiv.org/abs/2410.02683). *1,242 pairs, derived.* +- [wassname/machiavelli](https://huggingface.co/datasets/wassname/machiavelli) - power, deception, and harm choices in human-written choose-your-adventure games, reshaped for LLM scoring without fine-tuning. [paper](https://arxiv.org/abs/2304.03279), [code](https://github.com/wassname/machiavelli_as_ds). *139,269 nodes, human.* +- [wassname/machiavelli_character_scenarios](https://huggingface.co/datasets/wassname/machiavelli_character_scenarios) - roleplay decision prompts selected for spread on social/moral labels (fairness, deception, manipulation, promises, spying). *566 prompts, derived.* ## Social norms and moral foundations -| Dataset | N | Src | Paper | What it tests | -|---|---|---|---|---| -| `*` [wassname/tiny-mfv](https://huggingface.co/datasets/wassname/tiny-mfv) · [code](https://github.com/wassname/tinymfv) | 264 x 3 configs | human | Clifford 2015 | Forced-choice 7-way moral-foundation probe; a fast steering eval. Being renamed "moral aliens" with a moral map and more datasets (`moral-aliens-instrument` branch). | -| [wassname/tiny-mcf-vignettes](https://huggingface.co/datasets/wassname/tiny-mcf-vignettes) | 126 + 51 | mix | Clifford 2015 | Expanded foundations probe: Clifford vignettes plus confound-clean sci-fi ones, with self/other and uphold/violate conditions. | -| `*` [wassname/moral_stories_foundations](https://huggingface.co/datasets/wassname/moral_stories_foundations) | 12k pairs / 24k completions | human | [2012.15738](https://arxiv.org/abs/2012.15738) | Foundation-labelled moral vs immoral action pairs. Matched TRAINING set for the tiny-mfv eval. | -| [wassname/social_chemistry_101](https://huggingface.co/datasets/wassname/social_chemistry_101) · [code](https://github.com/mbforbes/social-chemistry-101) | 355,922 RoTs | human | [2011.00620](https://arxiv.org/abs/2011.00620) | Crowd-written rules-of-thumb over everyday situations, with social-acceptability and moral-foundation judgements. | -| `gh` [peterkirgis/llm-moral-foundations](https://github.com/peterkirgis/llm-moral-foundations) | repo | human | – | Eliciting moral foundations in frontier LLMs using vignettes. | +- `*` [wassname/tiny-mfv](https://huggingface.co/datasets/wassname/tiny-mfv) - forced-choice 7-way moral-foundation probe; a fast steering eval. Being renamed "moral aliens" with a moral map and more datasets (`moral-aliens-instrument` branch). [code](https://github.com/wassname/tinymfv). *264 x 3 configs, human (Clifford 2015).* +- [wassname/tiny-mcf-vignettes](https://huggingface.co/datasets/wassname/tiny-mcf-vignettes) - expanded foundations probe: Clifford vignettes plus confound-clean sci-fi ones, with self/other and uphold/violate conditions. *126 + 51, mix.* +- `*` [wassname/moral_stories_foundations](https://huggingface.co/datasets/wassname/moral_stories_foundations) - foundation-labelled moral vs immoral action pairs. Matched training set for the tiny-mfv eval. [paper](https://arxiv.org/abs/2012.15738). *12k pairs, human.* +- [wassname/social_chemistry_101](https://huggingface.co/datasets/wassname/social_chemistry_101) - crowd-written rules-of-thumb over everyday situations, with social-acceptability and moral-foundation judgements. [paper](https://arxiv.org/abs/2011.00620), [code](https://github.com/mbforbes/social-chemistry-101). *355,922 RoTs, human.* +- `gh` [peterkirgis/llm-moral-foundations](https://github.com/peterkirgis/llm-moral-foundations) - eliciting moral foundations in frontier LLMs using vignettes. ## Ethics judgements -| Dataset | N | Src | Paper | What it tests | -|---|---|---|---|---| -| [wassname/ethics_expression_preferences](https://huggingface.co/datasets/wassname/ethics_expression_preferences) | ~45k pairs | human | [2008.02275](https://arxiv.org/abs/2008.02275) | The ETHICS dataset (commonsense, deontology, justice, utilitarianism) as DPO pairs, expression form. | -| [wassname/ethics_qna_preferences](https://huggingface.co/datasets/wassname/ethics_qna_preferences) | ~113k pairs | human | [2008.02275](https://arxiv.org/abs/2008.02275) | Same ETHICS coverage (plus virtue) as question-and-answer DPO pairs. | -| [yixionghao/AEP_OOD_evaluation](https://huggingface.co/datasets/yixionghao/AEP_OOD_evaluation) | raw files | AI | – | OOD eval over safety traits (honesty, sycophancy, corrigibility, awareness, refusal, power-seeking) and Big-Five, with LLM-generated prompts. Non-standard layout (`choice-qa/` and `open-ended/` folders, no plain `load_dataset`); unvetted here. | -| [lcalvobartolome/fever_dplace_q](https://huggingface.co/datasets/lcalvobartolome/fever_dplace_q) | 185 | mix | – | Merges FEVER and D-PLACE to study entailment, contradiction, and cross-cultural value discrepancy. | +- [wassname/ethics_expression_preferences](https://huggingface.co/datasets/wassname/ethics_expression_preferences) - the ETHICS dataset (commonsense, deontology, justice, utilitarianism) as DPO pairs, expression form. [paper](https://arxiv.org/abs/2008.02275). *~45k pairs, human.* +- [wassname/ethics_qna_preferences](https://huggingface.co/datasets/wassname/ethics_qna_preferences) - same ETHICS coverage (plus virtue) as question-and-answer DPO pairs. [paper](https://arxiv.org/abs/2008.02275). *~113k pairs, human.* +- [yixionghao/AEP_OOD_evaluation](https://huggingface.co/datasets/yixionghao/AEP_OOD_evaluation) - OOD eval over safety traits (honesty, sycophancy, corrigibility, awareness, refusal, power-seeking) and Big-Five, with LLM-generated prompts. Non-standard layout (`choice-qa/` and `open-ended/` folders, no plain `load_dataset`); unvetted here. *raw files, AI.* +- [lcalvobartolome/fever_dplace_q](https://huggingface.co/datasets/lcalvobartolome/fever_dplace_q) - merges FEVER and D-PLACE to study entailment, contradiction, and cross-cultural value discrepancy. *185, mix.* ## Honesty, truthfulness, sycophancy -| Dataset | N | Src | Paper | What it tests | -|---|---|---|---|---| -| `gh` [meg-tong/sycophancy-eval](https://github.com/meg-tong/sycophancy-eval) | repo | mix | [2310.13548](https://arxiv.org/abs/2310.13548) | The Sharma et al. sycophancy probes (feedback, answer, mimicry). | -| [wassname/hh-rlhf-sycophantic](https://huggingface.co/datasets/wassname/hh-rlhf-sycophantic) | 5,964 pairs | mix | [2310.13548](https://arxiv.org/abs/2310.13548) | hh-rlhf pairs scored for how much more sycophantic the chosen response is; a knob to amplify sycophancy. | -| [wassname/truthful_qa_v2](https://huggingface.co/datasets/wassname/truthful_qa_v2) · [code](https://github.com/sylinrl/TruthfulQA) | 790 / 1,580 binary | human | [2109.07958](https://arxiv.org/abs/2109.07958) | The improved two-option multiple-choice TruthfulQA (common misconceptions). | -| [wassname/truthful_qa_preferences](https://huggingface.co/datasets/wassname/truthful_qa_preferences) | 817 | human | [2109.07958](https://arxiv.org/abs/2109.07958) | TruthfulQA cast as preference pairs. | -| `*` [wassname/genies_preferences](https://huggingface.co/datasets/wassname/genies_preferences) · [code](https://github.com/Joshuaclymer/GENIES) | 59 configs / 118,106 pairs | mix | [2311.07723](https://arxiv.org/abs/2311.07723) | An overlooked OOD testbed: 59 train-to-test distribution shifts for measuring how reward-model preferences generalise. | -| [unalignment/toxic-dpo-v0.1](https://huggingface.co/datasets/unalignment/toxic-dpo-v0.1) | 302 pairs | AI | – | Toxic vs safe DPO pairs; shows how few examples can de-align a model (gated). | +- `gh` [meg-tong/sycophancy-eval](https://github.com/meg-tong/sycophancy-eval) - the Sharma et al. sycophancy probes (feedback, answer, mimicry). [paper](https://arxiv.org/abs/2310.13548). +- [wassname/hh-rlhf-sycophantic](https://huggingface.co/datasets/wassname/hh-rlhf-sycophantic) - hh-rlhf pairs scored for how much more sycophantic the chosen response is; a knob to amplify sycophancy. [paper](https://arxiv.org/abs/2310.13548). *5,964 pairs, mix.* +- [wassname/truthful_qa_v2](https://huggingface.co/datasets/wassname/truthful_qa_v2) - the improved two-option multiple-choice TruthfulQA (common misconceptions). [paper](https://arxiv.org/abs/2109.07958), [code](https://github.com/sylinrl/TruthfulQA). *790 / 1,580 binary, human.* +- [wassname/truthful_qa_preferences](https://huggingface.co/datasets/wassname/truthful_qa_preferences) - TruthfulQA cast as preference pairs. [paper](https://arxiv.org/abs/2109.07958). *817, human.* +- `*` [wassname/genies_preferences](https://huggingface.co/datasets/wassname/genies_preferences) - an overlooked OOD testbed: 59 train-to-test distribution shifts for measuring how reward-model preferences generalise. [paper](https://arxiv.org/abs/2311.07723), [code](https://github.com/Joshuaclymer/GENIES). *59 configs / 118,106 pairs, mix.* +- [unalignment/toxic-dpo-v0.1](https://huggingface.co/datasets/unalignment/toxic-dpo-v0.1) - toxic vs safe DPO pairs; shows how few examples can de-align a model (gated). *302 pairs, AI.* ## Value orientations and personas -| Dataset | N | Src | Paper | What it tests | -|---|---|---|---|---| -| `gh` [ValueByte-AI/ValueBench](https://github.com/ValueByte-AI/ValueBench) | repo | human | ACL 2024 | Value-orientation eval drawn from established psychometric inventories. | -| [Anthropic/model-written-evals](https://huggingface.co/datasets/Anthropic/model-written-evals) | 3,252 | AI | [2212.09251](https://arxiv.org/abs/2212.09251) | LM-generated evals for persona, values, and ethics (Perez et al.). | -| [wassname/persona-steering-template-library](https://huggingface.co/datasets/wassname/persona-steering-template-library) · [code](https://github.com/wassname/persona-steering-template-library) | 400 | mix | – | Scored persona/template pairs, rating whether a template moves the intended value axis without off-axis confounds. | -| [wassname/speechmap-questions](https://huggingface.co/datasets/wassname/speechmap-questions) | 1,096 q / 144,459 resp | AI | – | Prompts and graded responses for probing where a model refuses or expresses values (speechmap.ai style). | -| [nvidia/Nemotron-Personas-USA](https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA) | 1,000,000 | AI | – | Synthetic personas grounded in US population distributions; a source pool for value-conditioned generation. | +- `gh` [ValueByte-AI/ValueBench](https://github.com/ValueByte-AI/ValueBench) - value-orientation eval drawn from established psychometric inventories (ACL 2024). +- [Anthropic/model-written-evals](https://huggingface.co/datasets/Anthropic/model-written-evals) - LM-generated evals for persona, values, and ethics (Perez et al.). [paper](https://arxiv.org/abs/2212.09251). *3,252, AI.* +- [wassname/persona-steering-template-library](https://huggingface.co/datasets/wassname/persona-steering-template-library) - scored persona/template pairs, rating whether a template moves the intended value axis without off-axis confounds. [code](https://github.com/wassname/persona-steering-template-library). *400, mix.* +- [wassname/speechmap-questions](https://huggingface.co/datasets/wassname/speechmap-questions) - prompts and graded responses for probing where a model refuses or expresses values (speechmap.ai style). *1,096 q / 144,459 resp, AI.* +- [nvidia/Nemotron-Personas-USA](https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA) - synthetic personas grounded in US population distributions; a source pool for value-conditioned generation. *1,000,000, AI.* ## Red-team and amoral contrast sets -| Dataset | N | Src | Paper | What it tests | -|---|---|---|---|---| -| [allenai/real-toxicity-prompts](https://huggingface.co/datasets/allenai/real-toxicity-prompts) | 99,442 | human | [2009.11462](https://arxiv.org/abs/2009.11462) | Web prompts scored for the risk of toxic continuations. | -| [TheDrummer/AmoralQA-v2](https://huggingface.co/datasets/TheDrummer/AmoralQA-v2) | n/a | AI | – | Amoral, uncensored QA pairs. | -| [soob3123/amoral_reasoning](https://huggingface.co/datasets/soob3123/amoral_reasoning) | n/a | AI | – | Amoral reasoning traces. | +- [allenai/real-toxicity-prompts](https://huggingface.co/datasets/allenai/real-toxicity-prompts) - web prompts scored for the risk of toxic continuations. [paper](https://arxiv.org/abs/2009.11462). *99,442, human.* +- [TheDrummer/AmoralQA-v2](https://huggingface.co/datasets/TheDrummer/AmoralQA-v2) - amoral, uncensored QA pairs. *AI.* +- [soob3123/amoral_reasoning](https://huggingface.co/datasets/soob3123/amoral_reasoning) - amoral reasoning traces. *AI.* ## Model organisms Models and datasets that deliberately sit off the modern, brand-safe alignment axis, useful as contrasts when measuring values. -| Resource | N | Src | What it is | -|---|---|---|---| -| [v2ray/4chan](https://huggingface.co/datasets/v2ray/4chan) | 50,835 | human | 4chan threads. Not "toxic" so much as edgy and offensive, focused on edge and offensive humour. Valuable precisely because it is almost the opposite of the harmless, brand-friendly training that frontier labs apply. | -| [wassname/v2ray_4chan_formatted](https://huggingface.co/datasets/wassname/v2ray_4chan_formatted) | 101,670 | human | The same 4chan corpus reformatted for LLM training/eval. | -| [talkie-lm/talkie-1930-13b-it](https://huggingface.co/talkie-lm/talkie-1930-13b-it) | model | model | A model trained on period-accurate 1930s text; a time-capsule organism whose moral and factual frame predates modern norms. | +- [v2ray/4chan](https://huggingface.co/datasets/v2ray/4chan) - 4chan threads. Not so much "toxic" as edgy and offensive, focused on edge and offensive humour. Valuable precisely because it is almost the opposite of the harmless, brand-friendly training that frontier labs apply. *50,835, human.* +- [wassname/v2ray_4chan_formatted](https://huggingface.co/datasets/wassname/v2ray_4chan_formatted) - the same 4chan corpus reformatted for LLM training/eval. *101,670, human.* +- [talkie-lm/talkie-1930-13b-it](https://huggingface.co/talkie-lm/talkie-1930-13b-it) - a model trained on period-accurate 1930s text; a time-capsule organism whose moral and factual frame predates modern norms. *model.* ## Upstream sources @@ -101,7 +90,7 @@ The original releases that several datasets above derive from. - `gh` [mbforbes/social-chemistry-101](https://github.com/mbforbes/social-chemistry-101) - Social Chemistry 101. - `gh` [Joshuaclymer/GENIES](https://github.com/Joshuaclymer/GENIES) - Generalization Analogies testbed. - `gh` [sylinrl/TruthfulQA](https://github.com/sylinrl/TruthfulQA) - TruthfulQA. -- `hf` [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) - helpful/harmless preferences and red-team transcripts. +- [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) - helpful/harmless preferences and red-team transcripts. ## Related lists and tools @@ -111,4 +100,4 @@ The original releases that several datasets above derive from. ## Contributing -Add an entry only if it has a public URL and a one-line description of what it evaluates. Note N, Src, and a paper where there is one, and prefer the canonical release. +Add an entry only if it has a public URL and a one-line description of what it evaluates. Note its row count and provenance, link a paper where there is one, and prefer the canonical release.