Gemma 4 (E2B/E4B) uses channel-based thinking tokens (<|think|>, <|channel>).
chat_template_extras() detects this via template string and passes
enable_thinking=False to all apply_chat_template calls in data gen,
dilemmas eval, and KL calib (via build_chat_text). Qwen3 and Gemma 3
return {} (existing thinking-mode handling unchanged).
The wassname/daily_dilemmas-self-honesty dataset uses paired-opposite
labels: if to_do has honesty in you_values -> to_do=+1, paired not_to_do=-1
even when not_to_do's own values are e.g. ['empathy'], unrelated to
honesty. Those 211/438 filler rows were dragging dW SI negative.
Changes:
- dilemmas.py: filter to action-specific honesty rows at load (227/438)
- data.py: narrow training pair to ('an honest',) / ('a dishonest',),
matching RepE extraction, simple_*_prompt baselines, and AntiPaSTO
honesty synonyms. Cluster paraphrases (candid/sincere/manipulative/...)
drift onto a broader axis -- comparison was unfair.
- prompt_baseline.py: regenerate engineered prompts via AxBench J.2
template with narrow-honesty concept slot. Old prompts brought in
integrity/transparency (POS) and Machiavellian/manipulation (NEG).
- nbs/rescore_honesty_only.py: ad-hoc rescore script confirming the
fix flips dW from -16 SI to +60 SI without rerunning the model.
Subagent review fixes:
- DataCfg / Cfg expose the grid directly (n_topics, n_personas, n_samples)
as required ints with paper defaults (20/5/10). Drops `n_pairs` and the
silent round() that made the count fuzzy. Drops `Optional[int]` smoke
overrides — smoke just sets 2/1/2 = 4 pairs.
- Drop hash()-based per-spec reseeding (process-nondeterministic via
PYTHONHASHSEED salt) and the `rng` parameter to _gen that never reached
model.generate. One torch.manual_seed at start; spec order seeded by rng.
- Delete _judge_filter stub + cfg.judge flag (dead code, paper §3 GPT-4.1-mini
filter not implemented yet — TODO comment instead).
- replicate._maybe_data: check len(ds) against n_topics × n_personas × n_samples
instead of n_pairs.
- justfile: drop --n-pairs 1000.
wassname