wassnameandClaudypoo 1d1365bd74 eval: add DEMO B free-reasoning pass (bs=1) alongside DEMO A forced readout ...

The forced-choice readout prefills the answer slot to read calibrated logprobs,
so it shows no real reasoning -- at think=1 its trace is just prompt + a token +
slot. Add free_generation_demo(): one bs=1 generation that lets the model think
to completion and answer naturally on a single vignette, same vignette+schema as
the readout. evaluate() now prints both (DEMO A relabelled, DEMO B new) via
loguru when verbose, and returns them in result['demos'] so callers get the text
without return_per_row. Free think budget = min(2048, max(512, max_think*batch))
-- bs=1 frees batch memory, floored so even think=1 reasons, capped for big batches.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>

2026-06-21 12:03:55 +08:00

..

tinymfv

eval: add DEMO B free-reasoning pass (bs=1) alongside DEMO A forced readout

2026-06-21 12:03:55 +08:00