mirror of
https://github.com/wassname/moral-maps.git
synced 2026-08-20 12:30:40 +08:00
Add shared gated_selectivity + si_flips metric
Canonical home (moralmaps.metrics) for the continuous coherence-gated selectivity sel_gated = (on - 0.1*off)*coh^2 plus the behavioral si_flips cross-check, imported (not re-forked) by steering-lite and j-steer so the definition can't silently diverge. README Measurement section defines both. 4 unit tests. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
@@ -148,46 +148,40 @@ The plotting code keeps only coefficients that every plotted dataset can still r
|
||||
|
||||
## Measurement
|
||||
|
||||
Steering is an intervention, so we judge it like surgery: did the intended thing move a lot, did everything else move as little as possible, and is the model still coherent? moralmaps reads three quantities that answer those, in rising order of steer-sensitivity.
|
||||
Steering is an intervention, so we judge it like surgery: did the intended factor move a lot, did everything else move as little as possible, and is the model still coherent? Four quantities, gated by coherence, in rising order of steer-sensitivity.
|
||||
|
||||
Coherence is measured with `pmass` is the share of probability the model puts on the valid answer tokens, and entropy is how spread-out the answer is within them:
|
||||
**Coherence — `pmass`** (the gate). The share of probability the model puts on the valid answer tokens (entropy is how spread-out the answer is within them):
|
||||
|
||||
$$m(c) = \mathbb{E}_i \sum_{a \in A_i} P_c(a \mid i)$$
|
||||
|
||||
where:
|
||||
where $c$ is the steering coefficient ($c=0$ is the base model), $i$ indexes items (a vignette or survey question), $A_i$ is the valid answer first-tokens for item $i$ (the seven foundation words, or scale points 1-5), and $P_c(a \mid i)$ is the model's next-token probability of answer token $a$ under steer $c$. A steer that drives `pmass` toward zero, or answers toward uniform, has broken the format — anything read off it is noise. It matters most on the *unintended* side: a steer that quietly turns answers to mush can look like change when it is really damage.
|
||||
|
||||
- $c$ is the steering coefficient ($c=0$ is the base model), and $m(c)$ is `pmass` at that coefficient
|
||||
- $i$ indexes items (a vignette or survey question)
|
||||
- $A_i$ is the set of valid answer first-tokens for item $i$ (the seven foundation words, or the scale points 1-5)
|
||||
- $P_c(a \mid i)$ is the model's next-token probability of answer token $a$ under steer $c$
|
||||
|
||||
A steer that drives `pmass` toward zero, or the answers toward uniform, has broken the format, and any value read off it is noise. Coherence matters most on the unintended side: a strong steer that quietly turns answers to mush can look like change when it is really damage.
|
||||
|
||||
The profile is what the maps plot: the human-comparable score per factor. For a survey it is the expected 1-5 answer (after reverse-keying); for MFV the mean forced-choice probability per foundation:
|
||||
**Profile** — what the maps plot: the human-comparable score per factor (expected 1-5 answer after reverse-keying for a survey, mean forced-choice probability per foundation for MFV):
|
||||
|
||||
$$\mathrm{profile}_d = \mathbb{E}_{i \in d}\sum_{k=1}^{M} k\,P(k \mid i) \qquad \mathrm{profile}_f = \mathbb{E}_i P(f \mid i)$$
|
||||
|
||||
where:
|
||||
($d$ a survey factor and $i \in d$ its items; $f$ an MFV foundation; $k$ a scale point $1..M$; $P(k \mid i)$ renormalized over $A_i$). It lands the model against human norms but *hides* steering: near a confident answer $E = \sum_k k\,p_k$ sits in a flat spot ($\partial E/\partial \ell_j = p_j (j - E) \to 0$ as $p_j$ concentrates), so a steer that only reallocates the tails barely moves it.
|
||||
|
||||
- $d$ is a survey factor (e.g. openness) and $i \in d$ its items; $f$ is an MFV foundation
|
||||
- $k$ is a scale point, from 1 to $M$ (here $M=5$)
|
||||
- $P(k \mid i)$ is the probability of answering scale point $k$ on item $i$, renormalized over $A_i$; $P(f \mid i)$ likewise for foundation $f$
|
||||
**Signal — $\Delta$** (the rank-centered logit contrast; `C` / `logit_contrast` in code, written $\Delta$ here to keep it off the coefficient $c$). Profile-shaped but in log-space with midpoint-centered weights, so its derivative is a fixed weight with no $p_j$ suppression — it still sees the steer when the profile is pinned:
|
||||
|
||||
This lands the model against human norms, but it hides steering: near a confident answer the expected score $E = \sum_k k\,p_k$ sits in a flat spot (its sensitivity to the answer logits, $\partial E/\partial \ell_j = p_j (j - E)$, vanishes as $p_j$ concentrates), so a steer that only reallocates the tails barely moves it. In the showcase CSVs this is the `mean` column; for MFV, model and human units differ, so the maps plot relative emphasis (each profile z-scored across foundations).
|
||||
$$\Delta_d(c) = \mathbb{E}_{i \in d}\sum_{k=1}^{M}\left(k - \tfrac{M+1}{2}\right)\ell_{i,k}^{(c)} \qquad \Delta_f = \mathbb{E}_i\left(\ell_{i,f}^{(+1)} - \ell_{i,f}^{(-1)}\right)$$
|
||||
|
||||
The steer signal is `C`, the rank-centered logit contrast: the same shape as the profile but in log-space with midpoint-centered weights, so its derivative is a fixed weight with no $p_j$ suppression and it still sees the steer when the profile is pinned:
|
||||
($\ell_{i,k}^{(c)}$ the logprob of scale-point $k$'s answer token at coefficient $c$, nats; $\ell_{i,f}$ likewise per foundation; $\Delta_f$ contrasts $c=+1$ vs $c=-1$).
|
||||
|
||||
$$C_d(c) = \mathbb{E}_{i \in d}\sum_{k=1}^{M}\left(k - \tfrac{M+1}{2}\right)\ell_{i,k}^{(c)} \qquad \Delta_f = \mathbb{E}_i\left(\ell_{i,f}^{(+1)} - \ell_{i,f}^{(-1)}\right)$$
|
||||
**Gated selectivity — `sel_gated`** (the headline). One base-anchored score that rewards the intended change, softly penalizes the unintended, and gates on coherence. Defined once in `moralmaps.metrics.gated_selectivity` and imported by every consumer (steering-lite, j-steer) so it cannot silently fork. On the per-foundation clr shift $\Delta_f = \mathrm{clr}_f(+C) - \mathrm{clr}_f(-C)$:
|
||||
|
||||
where:
|
||||
$$\mathrm{sel\_gated} = \Big(\underbrace{\tfrac{1}{|I|}\textstyle\sum_{f \in I} s_f\,\Delta_f}_{\text{on}} \;-\; \lambda\underbrace{\tfrac{1}{|O|}\textstyle\sum_{f \in O} |\Delta_f|}_{\text{off}}\Big)\cdot \mathrm{coh}^2, \qquad \mathrm{coh} = \min\!\Big(1,\ \frac{\min(\mathrm{pmass}_{+C},\,\mathrm{pmass}_{-C})}{\mathrm{pmass}_{\text{base}}}\Big)$$
|
||||
|
||||
- $\ell_{i,k}^{(c)}$ is the logprob of scale point $k$'s answer token on item $i$ at steering coefficient $c$ (nats); $\ell_{i,f}$ likewise for foundation $f$
|
||||
- $k - \tfrac{M+1}{2}$ is the midpoint-centered weight (for $M=5$: $-2,-1,0,1,2$)
|
||||
- $\Delta_f$ contrasts the full positive and negative steers, $c=+1$ vs $c=-1$
|
||||
where $I$ is the intended on-axis with signs $s_f \in \{+1,-1\}$ (e.g. $\{\text{authority}:-1,\ \text{care}:+1\}$ for an Authority-down / Care-up steer, or $\{\text{authority}:+1\}$ for a single clean axis), and $O$ is every other foundation (off-axis collateral, incl. social).
|
||||
|
||||
The intended change is $C$ (or $\Delta_f$) on the steered factor; the unintended change is $C$ moving on the other factors. A surgical steer has large intended change and small off-target change, at unchanged coherence.
|
||||
- **on** and **off** are both per-foundation-scale means, so $\lambda$ is a clean per-foundation trade.
|
||||
- $\lambda = 0.1$ (`OFF_WEIGHT`): off-axis is a soft *preference*, not co-equal. Moving the target the wrong way is a negative **on** at full weight; collateral is $|\Delta|$ at weight $\lambda$. At $\lambda=1$ the argmax-best "steer" is doing nothing (on$\approx$off$\approx$0 beats any real intervention with side effects).
|
||||
- **coherence** is a one-sided *squared* barrier on the worst arm: $=1$ when the format holds in both directions, $\to 0$ when steering turns answers to mush. It never rewards exceeding base coherence.
|
||||
- 95% bootstrap CI over vignette rows (2000×, seed 0), gated to match the point estimate.
|
||||
|
||||
We call the combined measurement surgical informedness: reward intended change, penalize unintended change, threshold on coherence. moralmaps reports the pieces (pmass, entropy, per-factor profile and $C$, and for MFV a nominal informedness, the Youden's J of the model's top foundation against the human top foundation); steering-lite folds them into the single base-anchored surgical informedness score it uses to rank steers.
|
||||
Because clr is pre-softmax nats, `sel_gated` is a direction-and-selectivity anchor for matched-KL comparison — **not** a behavioral effect size (a logit $8\to10$ at $p\approx1$ moves clr but changes no behavior).
|
||||
|
||||
**Flip informedness — `si_flips`** (the behavioral cross-check). The softmax-space companion `sel_gated` cannot give: the signed change in the model's forced-choice *pick* rate (argmax over clr, i.e. the actual answer) for the on-axis foundations, $\tfrac{1}{|I|}\sum_{f\in I} s_f\,[\Pr(\text{pick}=f\mid +C) - \Pr(\text{pick}=f\mid -C)]$. Bounded $[-1,1]$, Youden-J-style, and it saturates where clr does not — so it reports whether behavior, not just internal evidence, moved. (`moralmaps.metrics.si_flips`.)
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
@@ -28,6 +28,7 @@ from .instrument import Instrument, InstrItem, per_item_categorical, reduce_nomi
|
||||
from .instruments import get as get_instrument, INSTRUMENTS, build_instrument
|
||||
from .read import read_items, resolve_answer_ids, build_user_content
|
||||
from .readouts import expected_score, logit_contrast, logodds_agree, entropy
|
||||
from .metrics import gated_selectivity, si_flips, clr_per_row, OFF_WEIGHT
|
||||
from .administer import administer
|
||||
|
||||
|
||||
@@ -48,6 +49,8 @@ __all__ = [
|
||||
"evaluate", "administer", "get_instrument", "read_items",
|
||||
# ordinal readouts (pure functions of the raw answer-token logprobs)
|
||||
"expected_score", "logit_contrast", "logodds_agree", "entropy",
|
||||
# headline steering metrics (shared canonical defs; imported by steering-lite + j-steer)
|
||||
"gated_selectivity", "si_flips", "clr_per_row", "OFF_WEIGHT",
|
||||
# types consumers build / subset
|
||||
"Instrument", "InstrItem", "EvalResult", "EvalRow", "EvalInfo", "reduce_nominal", "reduce_ordinal",
|
||||
# data API
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Two headline steering metrics, defined ONCE here and imported by every consumer
|
||||
(steering-lite, j-steer) so a subtle method can't silently fork across repos.
|
||||
|
||||
Both read the per-foundation centered-log-ratio (clr) that `readouts.clr` builds from
|
||||
`per_row['score']` (the pre-softmax BMA'd + fwd/rev-averaged evidence, unbounded nats).
|
||||
The split is continuous-vs-flip -- they answer different questions:
|
||||
|
||||
gated_selectivity PRIMARY, continuous. Reward on-axis clr movement, softly penalize
|
||||
off-axis collateral, gate on coherence. No thresholds, no flips.
|
||||
A direction+selectivity anchor for matched-KL comparison -- because
|
||||
clr is pre-softmax nats, its MAGNITUDE is NOT a behavioral effect
|
||||
size (a logit 8->10 at p~=1 moves clr but changes no behavior).
|
||||
|
||||
si_flips SECONDARY, behavioral. Signed change in the model's forced-choice
|
||||
PICK rate (argmax over clr == argmax score == the answer) for the
|
||||
on-axis foundations. Bounded [-1, 1], softmax-space -- the behavioral
|
||||
cross-check the unbounded clr magnitude cannot give.
|
||||
|
||||
Everything the primary needs to be recomputed offline is returned in its result dict
|
||||
(on, off, both pmass arms + base, per-foundation Delta, CI, n_keys, intent, off_weight).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .guided import _DEFAULT_FORCED_FOUNDATIONS
|
||||
from .readouts import clr
|
||||
|
||||
FOUNDATIONS = tuple(_DEFAULT_FORCED_FOUNDATIONS) # care fairness loyalty authority sanctity liberty social
|
||||
|
||||
# off-axis is a soft PREFERENCE, not co-equal with on-axis (wassname). Two reasons it is
|
||||
# down-weighted rather than subtracted 1:1: (a) moving the target the WRONG way is worse than
|
||||
# collateral -- the wrong-way case is a negative `on` at full weight, collateral is |Delta| in
|
||||
# `off`; (b) at weight 1 the argmax-best "steer" is NONE (a do-nothing steer has on~off~0 and
|
||||
# beats any real intervention that has side effects). 0.1 = wassname's stated weight; one place.
|
||||
OFF_WEIGHT = 0.1
|
||||
|
||||
|
||||
def clr_per_row(report) -> dict[str, dict[str, float]]:
|
||||
"""Unclamped per-foundation clr from `per_row['score']` (pre-softmax nats). key = 'vid|cond'.
|
||||
|
||||
Gauge-free and non-competitive (clr_f = score_f - mean_j score_j): a shift in one
|
||||
foundation's evidence maps to that foundation and spreads only -1/K onto each other,
|
||||
unlike one-vs-rest log-odds whose logsumexp is dominated by the top foundation and
|
||||
fabricates off-axis collateral. Unclamped, so no +-6.9 clip censoring.
|
||||
"""
|
||||
out: dict[str, dict[str, float]] = {}
|
||||
for r in report["per_row"]:
|
||||
v = clr([float(x) for x in r["score"]])
|
||||
out[f"{r['id']}|{r['condition']}"] = {f: float(v[i]) for i, f in enumerate(FOUNDATIONS)}
|
||||
return out
|
||||
|
||||
|
||||
def _delta_per_f(pos_clr, neg_clr, keys) -> dict[str, float]:
|
||||
"""mean_row [clr_f(pos) - clr_f(neg)] per foundation, over the shared vignette-row keys."""
|
||||
n = len(keys)
|
||||
return {f: sum(pos_clr[k][f] - neg_clr[k][f] for k in keys) / n for f in FOUNDATIONS}
|
||||
|
||||
|
||||
def _on_off(delta: dict[str, float], intent: dict[str, int], off_set, off_weight) -> tuple[float, float, float]:
|
||||
"""on = mean_f∈intent intent[f]*Delta_f ; off = mean_f∉intent |Delta_f| ; sel = on - w*off.
|
||||
|
||||
on and off are both per-foundation-scale means, so the off_weight is a clean per-foundation
|
||||
trade (not confounded by how many foundations are on- vs off-axis)."""
|
||||
on = sum(intent[f] * delta[f] for f in intent) / len(intent)
|
||||
off = sum(abs(delta[f]) for f in off_set) / len(off_set)
|
||||
return on, off, on - off_weight * off
|
||||
|
||||
|
||||
def gated_selectivity(
|
||||
pos_clr: dict, neg_clr: dict, intent: dict[str, int], *,
|
||||
pmass_pos: float, pmass_neg: float, pmass_base: float,
|
||||
off_weight: float = OFF_WEIGHT, n_boot: int = 2000,
|
||||
) -> dict:
|
||||
"""PRIMARY metric. sel_gated = (on - off_weight*off) * coherence**2, 95% bootstrap CI over rows.
|
||||
|
||||
intent: {foundation_lower: +-1} the on-axis foundations and their intended signs, e.g.
|
||||
{'authority': -1, 'care': +1} for an Authority-down / Care-up steer, or {'authority': +1}
|
||||
for a single clean axis. Every other foundation is off-axis collateral.
|
||||
on = mean_f∈intent intent[f] * mean_row (clr_f(pos) - clr_f(neg)) -- + = moved toward intent.
|
||||
off = mean_f∉intent |mean_row (clr_f(pos) - clr_f(neg))| -- collateral, either way.
|
||||
coherence = min(1, min(pmass_pos, pmass_neg) / pmass_base): a ONE-SIDED squared barrier. Uses
|
||||
the WORST arm (a steer must stay in-format in BOTH directions); clamped at 1 so exceeding
|
||||
base coherence is never rewarded; -> 0 kills credit when steering breaks the answer format.
|
||||
|
||||
clr is pre-softmax nats: sel_gated is a direction+selectivity anchor, NOT a behavioral effect
|
||||
size. Pair it with si_flips for the behavioral claim.
|
||||
"""
|
||||
keys = [k for k in pos_clr if k in neg_clr]
|
||||
off_set = [f for f in FOUNDATIONS if f not in intent]
|
||||
coh = min(1.0, min(pmass_pos, pmass_neg) / pmass_base)
|
||||
coh2 = coh ** 2
|
||||
|
||||
delta = _delta_per_f(pos_clr, neg_clr, keys)
|
||||
on, off, sel = _on_off(delta, intent, off_set, off_weight)
|
||||
|
||||
rng = np.random.default_rng(0) # same seed/resampling as administer._ci, so CIs are comparable
|
||||
kk = np.array(keys)
|
||||
boot = np.array([
|
||||
_on_off(_delta_per_f(pos_clr, neg_clr, rng.choice(kk, len(kk), replace=True)),
|
||||
intent, off_set, off_weight)[2]
|
||||
for _ in range(n_boot)
|
||||
])
|
||||
# CI gated to match the point estimate (jsteer currently leaves its CI ungated; reconcile on migrate).
|
||||
lo, hi = float(np.percentile(boot, 2.5)) * coh2, float(np.percentile(boot, 97.5)) * coh2
|
||||
|
||||
return {
|
||||
"sel_gated": sel * coh2, "ci_lo": lo, "ci_hi": hi,
|
||||
"on": on, "off": off, "sel": sel,
|
||||
"coherence": coh, "off_weight": off_weight,
|
||||
"pmass_pos": pmass_pos, "pmass_neg": pmass_neg, "pmass_base": pmass_base,
|
||||
"delta_per_foundation": delta, "intent": dict(intent), "n_keys": len(keys),
|
||||
}
|
||||
|
||||
|
||||
def _pick(clr_row: dict[str, float]) -> str:
|
||||
"""The model's forced-choice answer for a row: argmax over clr (== argmax score == argmax p)."""
|
||||
return max(clr_row, key=clr_row.get)
|
||||
|
||||
|
||||
def si_flips(pos_clr: dict, neg_clr: dict, intent: dict[str, int]) -> dict:
|
||||
"""SECONDARY metric. Signed change in the forced-choice PICK rate for the on-axis foundations.
|
||||
|
||||
For each on-axis f: intent[f] * (rate(pick==f | pos) - rate(pick==f | neg)), averaged over
|
||||
the on-axis. `pick` is the argmax foundation (the actual answer), so this is a bounded,
|
||||
softmax-space, behavioral readout: + = the steer moved the CHOSEN foundation toward intent.
|
||||
Range [-1, 1]. Youden-J-style (a difference of rates); saturates where clr does not, which is
|
||||
exactly why it is the behavioral cross-check to the unbounded clr selectivity.
|
||||
"""
|
||||
keys = [k for k in pos_clr if k in neg_clr]
|
||||
n = len(keys)
|
||||
per_f: dict[str, float] = {}
|
||||
for f, s in intent.items():
|
||||
rate_pos = sum(_pick(pos_clr[k]) == f for k in keys) / n
|
||||
rate_neg = sum(_pick(neg_clr[k]) == f for k in keys) / n
|
||||
per_f[f] = s * (rate_pos - rate_neg)
|
||||
return {"si_flips": sum(per_f.values()) / len(per_f), "per_foundation": per_f, "n_keys": n}
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Unit guard for the two shared steering metrics (moralmaps.metrics). No model.
|
||||
|
||||
Builds synthetic pos/neg clr rows for a selective Authority-down / Care-up steer and
|
||||
checks: on>0, off small, the off_weight trade, the squared coherence barrier, and that
|
||||
si_flips is a bounded behavioral pick-rate change. -- Claude
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from moralmaps.metrics import FOUNDATIONS, gated_selectivity, si_flips, clr_per_row
|
||||
|
||||
INTENT = {"authority": -1, "care": +1} # steer Authority down, Care up
|
||||
|
||||
|
||||
def _row(auth: float, care: float) -> dict[str, float]:
|
||||
"""A clr row: authority/care set, others 0. (clr rows need not sum to 0 for the metric,
|
||||
which only reads per-foundation Deltas; realistic rows are centered but that is irrelevant here.)"""
|
||||
d = {f: 0.0 for f in FOUNDATIONS}
|
||||
d["authority"] = auth
|
||||
d["care"] = care
|
||||
return d
|
||||
|
||||
|
||||
def _sweep(auth_p, care_p, auth_n, care_n, others_p=0.0, others_n=0.0, n=8):
|
||||
"""n paired rows. pos = intended pole (auth low, care high), neg = opposite pole."""
|
||||
pos, neg = {}, {}
|
||||
for i in range(n):
|
||||
p = _row(auth_p, care_p)
|
||||
m = _row(auth_n, care_n)
|
||||
for f in FOUNDATIONS:
|
||||
if f not in INTENT:
|
||||
p[f] = others_p
|
||||
m[f] = others_n
|
||||
pos[f"vid{i}|other_violate"] = p
|
||||
neg[f"vid{i}|other_violate"] = m
|
||||
return pos, neg
|
||||
|
||||
|
||||
def test_selective_steer_scores_high():
|
||||
# pos drives auth DOWN (-2) and care UP (+2); neg is the mirror; off-axis flat.
|
||||
pos, neg = _sweep(auth_p=-2.0, care_p=+2.0, auth_n=+2.0, care_n=-2.0)
|
||||
r = gated_selectivity(pos, neg, INTENT, pmass_pos=0.9, pmass_neg=0.9, pmass_base=0.9)
|
||||
# on = mean[ -1*(-2 - +2), +1*(+2 - -2) ] / 2 = mean[+4, +4] = +4 ; off = 0
|
||||
assert abs(r["on"] - 4.0) < 1e-9
|
||||
assert r["off"] < 1e-9
|
||||
assert r["coherence"] == 1.0 # pmass preserved -> no discount
|
||||
assert abs(r["sel_gated"] - 4.0) < 1e-9
|
||||
assert r["ci_lo"] == r["ci_hi"] == 4.0 # zero row variance -> tight CI
|
||||
|
||||
|
||||
def test_off_axis_is_soft_preference():
|
||||
"""A sloppy steer (equal collateral on every off foundation) must still beat doing nothing,
|
||||
and only lose a little to a surgical steer -- off is weighted 0.1."""
|
||||
surgical, neg = _sweep(auth_p=-2.0, care_p=+2.0, auth_n=+2.0, care_n=-2.0, others_p=0.0)
|
||||
sloppy, neg2 = _sweep(auth_p=-2.0, care_p=+2.0, auth_n=+2.0, care_n=-2.0, others_p=4.0, others_n=0.0)
|
||||
rs = gated_selectivity(surgical, neg, INTENT, pmass_pos=0.9, pmass_neg=0.9, pmass_base=0.9)
|
||||
rl = gated_selectivity(sloppy, neg2, INTENT, pmass_pos=0.9, pmass_neg=0.9, pmass_base=0.9)
|
||||
assert rl["off"] > 0 # collateral registered
|
||||
assert rl["sel_gated"] > 0 # still beats inaction (sel=0)
|
||||
assert rs["sel_gated"] > rl["sel_gated"] # surgical wins
|
||||
assert (rs["sel_gated"] - rl["sel_gated"]) < 0.5 * rs["sel_gated"] # but only mildly (0.1 weight)
|
||||
|
||||
|
||||
def test_coherence_barrier_squared():
|
||||
pos, neg = _sweep(auth_p=-2.0, care_p=+2.0, auth_n=+2.0, care_n=-2.0)
|
||||
# worst arm pmass halves vs base -> coherence 0.5 -> sel scaled by 0.25.
|
||||
r = gated_selectivity(pos, neg, INTENT, pmass_pos=0.45, pmass_neg=0.9, pmass_base=0.9)
|
||||
assert abs(r["coherence"] - 0.5) < 1e-9
|
||||
assert abs(r["sel_gated"] - 4.0 * 0.25) < 1e-9
|
||||
# exceeding base coherence never rewards (clamped at 1).
|
||||
r2 = gated_selectivity(pos, neg, INTENT, pmass_pos=0.99, pmass_neg=0.99, pmass_base=0.9)
|
||||
assert r2["coherence"] == 1.0
|
||||
|
||||
|
||||
def test_si_flips_behavioral_bounded():
|
||||
# pos makes care the argmax pick, neg makes authority the pick.
|
||||
pos, neg = _sweep(auth_p=-2.0, care_p=+3.0, auth_n=+3.0, care_n=-2.0)
|
||||
r = si_flips(pos, neg, INTENT)
|
||||
# care: intent +1, rate(pos)=1, rate(neg)=0 -> +1 ; authority: intent -1, rate(pos)=0,
|
||||
# rate(neg)=1 -> -1*(0-1)=+1 ; mean = +1.
|
||||
assert abs(r["si_flips"] - 1.0) < 1e-9
|
||||
assert -1.0 <= r["si_flips"] <= 1.0
|
||||
Reference in New Issue
Block a user