mirror of
https://github.com/wassname/moral-maps.git
synced 2026-08-29 11:22:33 +08:00
wassnameandClaudypoo
fe6815a9c7
Replace the single forced-choice sampling reader (1 bit/call, truncated verbose models to empty at max_tokens=32, positionally biased) with a dense readout: rate every option 1-5 as JSON, N samples, binary items order-balanced to cancel positional bias, ratings mapped back to canonical order and normalized to a distribution. All of a model's calls fire concurrently (asyncio.gather over the async openrouter_wrapper; per_call=1 since providers ignore n>1). Each model carries a bootstrap 95% CI (over items + samples) drawn as error bars, so mushy models (deepseek-flash was near coin-flip) read as uncertain, not confident dots. One flaky provider is skipped, not fatal. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>