add v2 candidate persona library

This commit is contained in:
wassname
2026-06-13 10:09:32 +08:00
parent 327985c456
commit 4e27617821
7 changed files with 136 additions and 1 deletions
+6
View File
@@ -43,3 +43,9 @@ Verification:
- `uv run python scripts/validate_persona_axes_openrouter.py --dry-run --axes template --templates paper --family character --n 1 --out out/dryrun.json` planned 60 pairs.
- `python3 -m py_compile scripts/validate_persona_axes_openrouter.py scripts/export_persona_template_stats.py` passed.
- HF file list contains README plus 6 data files.
V2 candidate expansion:
- Added 16 candidate persona pairs, 12 candidate templates, and 12 candidate scenarios.
- Patched `--axes` to accept a persona-pair JSONL path.
- `uv run python scripts/validate_persona_axes_openrouter.py --dry-run --axes data/persona_pairs_v2_candidates.jsonl --templates data/templates_v2_candidates.txt --family data/scenarios_v2_candidates.jsonl --n 2 --out out/v2_candidates_dryrun.json` planned 384 pairs.
+49
View File
@@ -0,0 +1,49 @@
# V2 Expansion Plan
V2 separates candidate library material from measured validation stats.
## Candidate Files
- `data/persona_pairs_v2_candidates.jsonl`: short mirrored persona pairs.
- `data/templates_v2_candidates.txt`: reusable `{persona}` templates.
- `data/scenarios_v2_candidates.jsonl`: small scenario pool for smoke and first sweeps.
## Measurement Rule
Do not promote a template or persona pair because it sounds good. Promote only measured template x persona-pair cells.
Minimum v2 promotion target:
- at least 4 scenarios for a template x persona-pair cell
- `strict_pass_rate >= 0.5`
- `mean_axis_delta >= 3`
- `mean_off_axis_problem <= 2`
- `mean_max_style_abs_delta <= 2`
- no persona echo or refusal/role-breaks
## First V2 Sweep
Use a small factorial sweep before fanning out:
```sh
uv run python scripts/validate_persona_axes_openrouter.py \
--axes data/persona_pairs_v2_candidates.jsonl \
--templates data/templates_v2_candidates.txt \
--family data/scenarios_v2_candidates.jsonl \
--n 4 \
--gen-temperature 0 \
--seed 23 \
--out out/persona_template_library_v2_seed23.json
```
Then export:
```sh
uv run python scripts/export_persona_template_stats.py \
out/persona_template_library_v2_seed23.json \
--out-prefix out/persona_template_library_v2_seed23
```
## Notes
Some pairs are likely style-confounded by construction, especially calibrated vs overconfident and truth-over-approval. Keep them as canaries unless the off-axis audit is clean.