mirror of
https://github.com/wassname/persona-steering-template-library.git
synced 2026-06-27 16:46:08 +08:00
add v2 candidate persona library
This commit is contained in:
@@ -43,3 +43,9 @@ Verification:
|
||||
- `uv run python scripts/validate_persona_axes_openrouter.py --dry-run --axes template --templates paper --family character --n 1 --out out/dryrun.json` planned 60 pairs.
|
||||
- `python3 -m py_compile scripts/validate_persona_axes_openrouter.py scripts/export_persona_template_stats.py` passed.
|
||||
- HF file list contains README plus 6 data files.
|
||||
|
||||
V2 candidate expansion:
|
||||
|
||||
- Added 16 candidate persona pairs, 12 candidate templates, and 12 candidate scenarios.
|
||||
- Patched `--axes` to accept a persona-pair JSONL path.
|
||||
- `uv run python scripts/validate_persona_axes_openrouter.py --dry-run --axes data/persona_pairs_v2_candidates.jsonl --templates data/templates_v2_candidates.txt --family data/scenarios_v2_candidates.jsonl --n 2 --out out/v2_candidates_dryrun.json` planned 384 pairs.
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# V2 Expansion Plan
|
||||
|
||||
V2 separates candidate library material from measured validation stats.
|
||||
|
||||
## Candidate Files
|
||||
|
||||
- `data/persona_pairs_v2_candidates.jsonl`: short mirrored persona pairs.
|
||||
- `data/templates_v2_candidates.txt`: reusable `{persona}` templates.
|
||||
- `data/scenarios_v2_candidates.jsonl`: small scenario pool for smoke and first sweeps.
|
||||
|
||||
## Measurement Rule
|
||||
|
||||
Do not promote a template or persona pair because it sounds good. Promote only measured template x persona-pair cells.
|
||||
|
||||
Minimum v2 promotion target:
|
||||
|
||||
- at least 4 scenarios for a template x persona-pair cell
|
||||
- `strict_pass_rate >= 0.5`
|
||||
- `mean_axis_delta >= 3`
|
||||
- `mean_off_axis_problem <= 2`
|
||||
- `mean_max_style_abs_delta <= 2`
|
||||
- no persona echo or refusal/role-breaks
|
||||
|
||||
## First V2 Sweep
|
||||
|
||||
Use a small factorial sweep before fanning out:
|
||||
|
||||
```sh
|
||||
uv run python scripts/validate_persona_axes_openrouter.py \
|
||||
--axes data/persona_pairs_v2_candidates.jsonl \
|
||||
--templates data/templates_v2_candidates.txt \
|
||||
--family data/scenarios_v2_candidates.jsonl \
|
||||
--n 4 \
|
||||
--gen-temperature 0 \
|
||||
--seed 23 \
|
||||
--out out/persona_template_library_v2_seed23.json
|
||||
```
|
||||
|
||||
Then export:
|
||||
|
||||
```sh
|
||||
uv run python scripts/export_persona_template_stats.py \
|
||||
out/persona_template_library_v2_seed23.json \
|
||||
--out-prefix out/persona_template_library_v2_seed23
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
Some pairs are likely style-confounded by construction, especially calibrated vs overconfident and truth-over-approval. Keep them as canaries unless the off-axis audit is clean.
|
||||
Reference in New Issue
Block a user