Commit Graph

Select branches

Hide Pull Requests

add-scenario-sources

add-w2s-character-axes-and-prompts

main

#1

#2

#3

Mono Color

• 5e177a2745 Merge pull request #3 from wassname/add-bullshit-benchmark-loader main wassname (Michael J Clark)andGitHub 2026-08-09 10:47:36 +08:00
• 8353897780 scenarios: register BullshitBench for credulity screening wassname 2026-07-26 10:23:55 +08:00
• 0dde39e8d0 test: cover BullshitBench scenario loading wassname 2026-07-26 10:23:49 +08:00
• 9639ae3c81 fix: repair scenario exporter paths after script move wassname 2026-07-26 10:23:44 +08:00
• 0b53320bdf scenarios: add the bullshit benchmark (55 incoherent-premise questions) wassnameandClaudypoo 2026-07-26 10:21:55 +08:00
• b8e22f42fb refactor: sort scripts/ by who runs it wassnameandClaudypoo 2026-07-25 10:23:22 +08:00
• 3593189142 refactor: replace 4 per-axis shell scripts with run_axis.py + data/axes/<id>/config.yaml wassnameandClaudypoo 2026-07-25 10:12:01 +08:00
• e7a9f6c7e3 refactor: drop orphaned prepare_authority script, rename export to be axis-neutral wassnameandClaudypoo 2026-07-25 10:11:53 +08:00
• a519fb8e39 chore: pin and ignore the Inspect model cache dir wassnameandClaudypoo 2026-07-25 10:11:45 +08:00
• 94d839629c Update trying_mean_mass_diff.md wassname (Michael J Clark)andGitHub 2026-07-24 21:04:30 +08:00
• df4979f926 Update trying_mean_mass_diff.md wassname (Michael J Clark)andGitHub 2026-07-24 21:00:20 +08:00
• 680fad5a31 Create trying_mean_mass_diff.md wassname (Michael J Clark)andGitHub 2026-07-24 16:32:27 +08:00
• 77e739a3cc docs(skill): rename scenario 'pool' to 'dataset', annotate Links with when-to-open wassnameandClaudypoo 2026-07-24 14:55:34 +08:00
• 7ea078f520 docs(skill): surface scenario pools from SKILL.md entry point wassnameandClaudypoo 2026-07-24 14:52:27 +08:00
• 59cae495b3 docs(scenarios): describe each data/scenarios pool with source links wassnameandClaudypoo 2026-07-24 14:51:52 +08:00
• 93d9ed165e personas: add government_authority pair (deference/disrespect to governmental authority) wassnameandClaudypoo 2026-07-19 13:32:45 +08:00
• a64f0438a0 Document Inspect validation outputs wassname 2026-07-18 21:14:00 +08:00
• dd218c8223 Point persona runbooks at the Inspect evaluator wassname 2026-07-18 19:09:19 +08:00
• 56a35d92a7 Run persona validation through Inspect AI wassname 2026-07-18 19:09:02 +08:00
• 64fafcf301 Remove prose repeated by the steering example wassname 2026-07-18 11:39:16 +08:00
• 81096c6d4c Show steering from paired prompts to inference wassname 2026-07-18 11:38:48 +08:00
• 90ccc40391 Introduce steering before the runbook wassname 2026-07-18 11:29:10 +08:00
• baac06a2be Replace Quarto site with concise README wassname 2026-07-18 10:49:02 +08:00
• bdebe3fb25 Simplify persona scenario and template selection wassname 2026-07-18 10:48:35 +08:00
• f353e73f1d Require N=1 subagent UAT for skill changes wassname 2026-07-18 09:54:50 +08:00
• 5375af2753 Move persona steering runbook to skill root wassname 2026-07-18 09:54:50 +08:00
• 4f0780f10f Docs: describe min_side/both-directions gate; drop stale 0.4 magic number wassnameandClaudypoo 2026-07-17 16:06:13 +08:00
• 9c36cffd52 SKILL.md: drop two orphaned bullets left over from deleted section wassnameandClaudypoo 2026-07-17 14:53:53 +08:00
• 591478fa30 SKILL.md: fold run-reading guidance into the procedure steps, cut standalone section wassnameandClaudypoo 2026-07-17 14:52:48 +08:00
• 554a425436 Docs: baseline-anchored judging in SKILL.md + README (metrics, gates, smoke test) wassnameandClaudypoo 2026-07-17 14:48:21 +08:00
• baceaa24e7 Fix export_selections drift from baseline-anchored validator (external review) wassnameandClaudypoo 2026-07-17 14:30:23 +08:00
• 8a092bafb0 Stage A v3 run scripts for honesty + credulity (baseline-anchored) wassnameandClaudypoo 2026-07-17 14:01:28 +08:00
• 57680db2c0 Baseline-anchor axis judging: gate on min side delta (neg < baseline < pos) wassnameandClaudypoo 2026-07-17 14:01:28 +08:00
• 958f3c7973 Export credulity strict-pass (24) + fix export for missing confound_judgment wassname 2026-07-06 07:37:43 +08:00
• 0096b87450 Add export_selections.py + honesty top-50 selections wassname 2026-07-06 06:12:03 +08:00
• a80a0d56ea Add overall_score + configurable thresholds + exclude-confound-dims wassname 2026-07-06 06:11:13 +08:00
• 0cd781f8b9 Add parse_stage_a.py: rank Stage A templates by strict_pass_rate then axis_delta wassname 2026-07-05 22:24:01 +08:00
• bde3105668 SKILL.md: make Stage A/B cost structure explicit (100x12 + 1x1000 << 100x1000) wassname 2026-07-05 22:20:36 +08:00
• 542da600c6 SKILL.md: note axes are orthogonal, run Stage A per-axis not cross-product wassname 2026-07-05 22:00:06 +08:00
• bbfc812f45 Add --n-per-source for stratified sampling + SKILL.md clarity wassname 2026-07-05 21:51:28 +08:00
• 48e5596730 Mirror honesty/credulity selections into library data/selections/ wassname 2026-07-05 13:44:26 +08:00
• 1cf18539e6 Add user-perspective notes from honesty/credulity work wassname 2026-07-05 13:41:29 +08:00
• 8ee98a0126 Per-axis persona jsonl + Stage B launcher for honesty/credulity wassname 2026-07-05 13:19:06 +08:00
• a949cd248e Thread provider_only to bounded axis judge; add Stage A template candidates wassname 2026-07-05 12:55:17 +08:00
• 6b3a4b062a Add bounded-thinking judge: phase-1 bounded thinking + phase-2 force-answer wassname 2026-07-05 12:53:24 +08:00
• 82c1599e7a Add credulous_skeptical axis pair for honesty x credulity steering map wassname 2026-07-05 12:38:38 +08:00
• f37cca55f3 Tighten pure authority selection wassname 2026-07-01 00:29:31 +08:00
• 6450d5140b Use grammar-neutral authority templates wassname 2026-06-30 20:02:35 +08:00
• f355b47ea9 Fix authority validation to one pure pair wassname 2026-06-30 19:51:04 +08:00
• e61f8a0598 Use wrapper idle-timeout retry wassname 2026-06-30 15:52:17 +08:00
• 6116a45d40 Use wrapper retry fix wassname 2026-06-30 15:25:48 +08:00
• 3e7341478d Disable Qwen thinking in persona validation wassname 2026-06-30 15:09:20 +08:00
• 0f1d35a24b Pin persona generator provider wassname 2026-06-30 15:02:00 +08:00
• 2863482427 Validate authority axes through wrapper wassname 2026-06-30 14:51:22 +08:00
• 52065c7d1b Shorten steering selection README note wassname 2026-06-30 10:57:57 +08:00
• bbbedf2ad4 Document strict steering selection workflow wassname 2026-06-30 10:55:33 +08:00
• 7ecdfea07e Add strict authority scenario selection wassname 2026-06-30 10:52:21 +08:00
• c489cf3a3e Filter selected authority scenarios by strict pass wassname 2026-06-30 10:51:36 +08:00
• b4225336da Add authority steering selection exporter wassname 2026-06-30 10:38:57 +08:00
• 4edabbc62d Add authority steering selection preparer wassname 2026-06-30 10:14:33 +08:00
• db1a3f8b47 repo: move scenario sources and committed results wassname 2026-06-30 09:45:32 +08:00
• a919adcb8a scenario_sources: add sycophancy genies and valuebench wassname 2026-06-30 09:19:02 +08:00
• 6d4b1d0bff templates: add formatting stress persona prompts wassname 2026-06-30 08:50:23 +08:00
• 488dc1c1c3 scenario_sources: add capped source scenario samples wassname 2026-06-30 08:48:51 +08:00
• 76d8fde10b scenario_sources: export machiavelli scenario jsonl wassname 2026-06-30 08:43:40 +08:00
• 97c1ac3b0b organize persona and scenario data paths wassname 2026-06-29 19:16:41 +08:00
• f69bbe6e09 machiavelli: publish full moral-contrast text set wassname 2026-06-29 18:40:31 +08:00
• fcd785c7da machiavelli: tighten refusal validation phrase match wassname 2026-06-29 06:44:01 +08:00
• 559f435f5f machiavelli: frame combo prompts as roleplay choices wassname 2026-06-29 05:17:25 +08:00
• beecf46d84 machiavelli: document deepseek summary prompt contract wassname 2026-06-29 05:11:54 +08:00
• 2a23886ec6 machiavelli: sort full moral dataset by subtle spread wassname 2026-06-29 05:03:00 +08:00
• b446589be7 Merge pull request #2 from wassname/add-scenario-sources wassname (Michael J Clark)andGitHub 2026-06-28 22:25:44 +08:00
• 0e2f331733 machiavelli: upload top moral-contrast scenarios add-scenario-sources wassname 2026-06-28 22:25:06 +08:00
• f7a94a2ddb machiavelli: add moral-contrast scorer handover wassname 2026-06-28 21:39:43 +08:00
• 11a43903f4 Add scenario_sources: public-dataset scenario loaders + screen workflow wassname-claudeandClaudypoo 2026-06-28 06:23:48 +00:00
• 9e73d9fa46 docs: align persona-template skill workflow wassname 2026-06-25 14:08:19 +08:00
• 8b99b2dca0 docs: shorten scenario suffix appendix wassname 2026-06-25 13:56:35 +08:00
• cd695c411b docs: improve quick-scroll README wassname 2026-06-25 13:36:00 +08:00
• 8162aa1ee9 docs: widen Quarto HTML layout wassname 2026-06-25 13:27:21 +08:00
• afbfbf514f docs: add interactive refusal tables wassname 2026-06-25 13:23:34 +08:00
• cfcb57b9ce docs: use one Quarto source for README and Pages wassname 2026-06-25 13:06:12 +08:00
• 024fb3d545 docs: track model matrix inputs for Pages render wassname 2026-06-25 12:45:58 +08:00
• bcbc1d0f65 docs: render Pages with Quarto workflow wassname 2026-06-25 12:44:39 +08:00
• 22dd2c2df9 docs: rank README result tables by t-stat wassname 2026-06-25 12:33:11 +08:00
• caceaebbf0 docs: streamline README and add interactive Pages plot wassname 2026-06-25 12:31:50 +08:00
• d31cac9068 docs: simplify model matrix visualization wassname 2026-06-25 12:20:35 +08:00
• 026b22e131 docs: simplify model matrix ranking wassname 2026-06-25 11:54:06 +08:00
• 2f62327acc docs: render README with Quarto wassname 2026-06-25 11:44:04 +08:00
• 026a57e246 docs: make README tables rerenderable wassname 2026-06-25 11:31:49 +08:00
• 2f7184f609 eval: summarize refusal probe model matrix wassname 2026-06-25 11:12:12 +08:00
• da435ccb67 eval: add refusal probe axes wassname 2026-06-25 10:30:33 +08:00
• a2b0bcbc76 eval: add roleplay context stress templates wassname 2026-06-25 10:24:20 +08:00
• 85b4a6f354 eval: refresh stress template results wassname 2026-06-25 09:58:23 +08:00
• fffab4e25a fix: normalize new stress templates wassname 2026-06-25 09:52:46 +08:00
• 3745b280f2 Update template_catalog.yaml wassname (Michael J Clark)andGitHub 2026-06-24 21:01:29 +08:00
• a88acae536 docs: add persona prior-art guide wassname 2026-06-23 10:32:20 +08:00
• 234ea38eda docs: add persona selection guide wassname 2026-06-23 10:17:36 +08:00
• 55321e6799 Merge pull request #1 from wassname/add-w2s-character-axes-and-prompts wassname (Michael J Clark)andGitHub 2026-06-21 13:10:00 +08:00
• 6b272b8c86 Make validator honor self-contained scenario prompts (fixes 3p suffix clash) add-w2s-character-axes-and-prompts wassname-claudeandClaudypoo 2026-06-21 04:25:13 +00:00
• 852c441762 Correct 1p speculation with tested result: first-person prompts make it worse wassname-claudeandClaudypoo 2026-06-21 04:10:15 +00:00