github.com/petergpt/bullshit-benchmark v2.0: confidently-posed nonsense across 10 techniques,
e.g. 'the default risk profile of our content strategy given the current engagement yield curve'.
Answering as if the premise held is sycophancy that needs no domain knowledge, so unlike
sycophancy_eval it does not confound agreeableness with ignorance -- there is no fact the model
could be failing to know. In j-steer-dev the wrong-answer nudge left the blunt pole essentially
unmoved (10 lost vs 7 gained, p=0.63) while moving the flattering pole hard (0.13 -> 0.31 flip
rate, p=0.008), so a family that discriminates the skeptical direction fills a real gap.
Each question ships a nonsensical_element, so a judge can be anchored on what to look for
rather than rating an abstraction; the benchmark's scale is 0 engaged / 1 hedged / 2 flagged.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Top level is now only what the runbook or a re-run touches: validate_persona_axes,
bounded_thinking_judge, template_catalog, export_selections, parse_stage_a, run_axis,
export_steering_selection. Corpus ingestion and publishing moved to scripts/corpus/,
plotting and stats to scripts/report/.
Moved files needed parents[1] -> parents[2]; the two corpus scripts that import
template_catalog use the sys.path shim bounded_thinking_judge_liveproof already used.
Also completes the export_steering_selection rename: an earlier git reset had dropped
the staged deletion, leaving both filenames tracked.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
wassname