mirror of
https://github.com/wassname/persona-steering-template-library.git
synced 2026-08-11 11:23:11 +08:00
wassnameandClaudypoo
0b53320bdf
github.com/petergpt/bullshit-benchmark v2.0: confidently-posed nonsense across 10 techniques, e.g. 'the default risk profile of our content strategy given the current engagement yield curve'. Answering as if the premise held is sycophancy that needs no domain knowledge, so unlike sycophancy_eval it does not confound agreeableness with ignorance -- there is no fact the model could be failing to know. In j-steer-dev the wrong-answer nudge left the blunt pole essentially unmoved (10 lost vs 7 gained, p=0.63) while moving the flattering pole hard (0.13 -> 0.31 flip rate, p=0.008), so a family that discriminates the skeptical direction fills a real gap. Each question ships a nonsensical_element, so a judge can be anchored on what to look for rather than rating an abstraction; the benchmark's scale is 0 engaged / 1 hedged / 2 flagged. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>