Stage A (bounded_thinking judge, 120 pairs, 0 errors, 0 judge-did-not-commit):
- truth_over_approval winner: 'Speak with the priorities of someone {persona}.' (axisD=3.00)
- credulous_skeptical winner: 'Judge the case as someone {persona} would.' (33.3% strict pass)
Stage B launched for both axes (n=50, 6 sources). The credulity steering-lite runner is
updated to its Stage A winner template.
Route the bounded-thinking axis judge to the same DeepInfra provider as the
generator (DeepInfra is known to support qwen3 thinking, per the liveproof), so
the reasoning judge does not land on a provider that ignores thinking params.
Port of the gist (wassname/72eed3a1ddfc286c5e12a118dfa30161) adapted to this repo's
openrouter_wrapper client (no inspect-ai dep). A reasoning judge that deliberates to
its max-token budget emits NO verdict; a parser defaulting to 0 silently launders that
non-conclusion into a tie indistinguishable from a real SCORE: 0. Fix:
- scripts/bounded_thinking_judge.py: phase-1 think at native params capped by
max_tokens=BUDGET; phase-2 if no verdict, continue the conversation with the truncated
reasoning + 'answer NOW' and thinking OFF (reasoning_effort=none). parse_score returns
(score, found) so a real 0 (found=True) is never confused with a non-verdict
(found=False). N-sample averaging (reproducibility from N, not OOD greedy temp0).
- validate_persona_axes_openrouter.py: opt-in --axis-judge-method {json,bounded_thinking}
(default json, unchanged for the authority baseline). The bounded path returns a JSON
string shaped like the json path so _pairwise_expected/_validate_axis_obj work unchanged;
found=False excludes the item from strict_pass (never laundered into a tie). SCORE -5..+5
maps to the 1..5 A_more_target_than_B scale as 3.0 + 0.4*SCORE.
- bounded_thinking_judge_liveproof.py: one real call on qwen3-14b via DeepInfra; confirmed
found=True score=-5 forced=False on a credulous_skeptical item (skeptical A correctly
judged less credulous than credulous B).
- data/personas/persona_pairs_honesty_credulity.jsonl: the two fixed axes for this work.
Loaders that turn public moral-judgment datasets into short self-contained
scenario PROMPTS (not completions) for persona pair generation, affordance-
filtered (no prescribed-content / refusal-trap / single-answer QA) and length-
capped: AIRiskDilemmas (AI-seat), moral_stories_foundations, daily_dilemmas,
social_chemistry_101, ethics_qna commonsense (3p judgment), plus machiavelli via
an offline deepseek-v4-flash compressor cached to data/ (raw obs ~350 words).
Moral-RolePlay skipped (fiction/eval-leak). README documents the affordance
contract + load->screen->keep workflow; HANDOVER lists remaining work (run the
screen, fold existing scenario jsonls, machiavelli HF dataset preserving labels).
Also harden validate_persona_axes_openrouter.py: guard resp.choices is None
(OpenRouter error bodies) so one bad API response can't abort the whole screen.
Ported from wassname/w2schar-mini.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The 52 3p-observer scenarios carry their own question + length
("...what does the actor do next? Answer in one or two sentences."),
but _generation_prompt always appended a 1p "What do you do? Answer in
80-140 words." -- two conflicting questions and a POV flip, so the data
could not reproduce the appendix's 3p measurement through this script
(caught in review).
Add a "self_contained" flag on scenario rows; when set, _generation_prompt
emits the scenario verbatim instead of appending its default 1p question.
Bare-vignette families without the flag are unchanged. README appendix
rewritten to lead with the runnable command and explain the flag.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
wassname