Commit Graph
13 Commits
Author SHA1 Message Date
wassnameandClaudypoo b726db958b fix: drop dangling mean_pmass_format key in evaluate() return (NameError); gitignore .pi/
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-06-18 22:03:28 +08:00
wassname fde697130d Merge branch 'main' of https://github.com/wassname/tinymfv 2026-06-18 21:32:07 +08:00
wassname 220208205c fix: package vignette data with tinymfv 2026-06-13 22:50:56 +08:00
wassname 726324e772 misc 2026-05-20 04:31:24 +00:00
wassname 8dfaf299ca rename 2026-05-08 15:30:06 +08:00
wassname b12770cb78 fixes, naming 2026-05-08 15:23:54 +08:00
wassname d796df85c8 improved to have better airisk, better eval that distinguished factors 2026-05-08 14:04:56 +08:00
wassname 076b859b9a fix: remove HuggingFace fallback from load_vignettes for strict fail-first behavior 2026-05-03 13:00:11 +08:00
wassname bcbdb9cc6f feat: multi-label moral foundation ratings with z-scored frame averaging and human calibration
- Add scripts/07_multilabel.py: LLM judge rates all 7 foundations per vignette
  using violation (forward) and acceptability (reverse) frames
- Foundation definitions drawn from Clifford et al. (2015) survey rubric
- Z-score each frame per foundation before averaging to cancel range bias
- Calibrate LLM Likert → human % via per-foundation OLS (classic set only)
- Add scripts/07a_merge_labels.py: merges llm_* and calibrated_* into vignette files
- Update README and HF dataset card with methodology and calibration quality table
- Classic set: 80.3% dominant-foundation accuracy, Pearson r 0.69-0.89 per foundation
2026-05-03 12:48:14 +08:00
wassname 881ac16c24 API improvements: rename clifford->classic, default load_vignettes to all, add dual-axis docs, and update HF upload script 2026-05-03 07:01:28 +08:00
wassname addf47c5a0 quiet pmass-low warning: one summary per batch
Was emitting `logger.warning("pmass=0.XX<0.9 — top-5: ...")` per-row, which
spammed the log heavily during heavy-steering eval (many rows go OOD at once).
Now collects all low-pmass rows in the batch and emits one summary line with
the worst-case top-5, e.g.:

    pmass<0.9 on 7/16 rows in this batch; worst=0.412 top-5: '1'=0.40, ...

Same diagnostic signal, ~16× fewer log lines per batch.
2026-05-03 06:50:19 +08:00
wassname 252e62abb7 decent 2026-04-30 21:22:07 +08:00
wassname a155f5594b valdiation 2026-04-30 20:08:12 +08:00