Unsupervised-Elicitation/nbs/04_REVIEW.md

Review: Evidence Weighting Notebook (04_test_evidence_weighting.py)

Date: 2025-10-09
Status: ✅ Runs successfully with warnings

---

✅ Execution Summary

• Runtime: ~30 seconds (10 cache builds + 30 predictions)
• Coverage: 8/10 targets predicted (2 targets not sampled)
• Output: 30 predictions saved to JSONL
• Best accuracy: 62.5% (5/8) with weights (0.4, 0.2, 0.1, 0.2, 0.1)

---

⚠️ Warnings Found

1. RuntimeWarning: overflow encountered in exp

Location: Lines 59, 277
Cause: raw_logprob_diff values are extreme (~±995) causing np.exp(-995) → 0
Impact: Score saturates to 0.0 or 1.0 (not a bug, but loses granularity)

# Line 277
score = 1 / (1 + np.exp(-raw_diff))  # raw_diff = 995 → exp(-995) = 0 → score = 1.0

Fix: Clip logprob diffs before sigmoid:

raw_diff_clipped = np.clip(raw_diff, -20, 20)  # exp(±20) is numerically safe
score = 1 / (1 + np.exp(-raw_diff_clipped))

---

📊 Results Analysis

Grid Search Findings

Weights (flip, conf, var, mp, cons)	Accuracy	Cal Error	Notes
(0.4, 0.2, 0.1, 0.2, 0.1)	62.5%	0.484	Heavy flip (BEST)
(0.5, 0.1, 0.1, 0.2, 0.1)	62.5%	0.478	Even heavier flip
(0.3, 0.2, 0.2, 0.2, 0.1)	50.0%	0.494	Original baseline
(0.2, 0.3, 0.2, 0.2, 0.1)	50.0%	0.500	More conf
(0.1, 0.4, 0.1, 0.3, 0.1)	50.0%	0.504	Conf + MP

Key observations:

• Flip sensitivity dominates: Higher flip weight (0.4-0.5) → better accuracy
• Variance/MP less useful: Reducing their weight doesn't hurt
• Small sample: 8 targets is noisy, but trend is clear

Ensemble Statistics (Best Weights)

UID	Mean	Std	Predicted	True	Correct
9	1.000	0.000	1	0	✗
0	0.643	0.226	1	0	✗
7	0.500	0.316	0	0	✓
3	0.500	0.000	0	0	✓
6	0.688	0.348	1	1	✓
5	0.500	0.000	0	1	✗
1	0.833	0.236	1	1	✓
2	0.750	0.250	1	1	✓

Patterns:

• High confidence wrong: uid=9 (mean=1.0, std=0.0) predicts 1 but true=0
• No variance = no info: uid=9, 3, 5 have std=0.0 (only 1-2 predictions)
• Moderate variance useful: uid=6 (std=0.348) is correct

---

🔍 Consistency Analysis

Data Structure (5 question groups × 2 contradictory claims)

Group 0 (watermelon seeds):
  uid=0, key=B, label=0: "You grow watermelon..." (FALSE)
  uid=1, key=A, label=1: "The watermelon passes..." (TRUE)
  
Group 1 (fortune cookies):
  uid=2, key=A, label=1: "The precise origin..." (TRUE)
  uid=3, key=B, label=0: "Fortune cookies originated in China" (FALSE)
  
Group 2 (veins):
  uid=4, key=B, label=0: "...deoxygenated blood" (FALSE)
  uid=5, key=A, label=1: "...blue light penetration" (TRUE)
  
Group 3 (chili pepper):
  uid=6, key=A, label=1: [Correct answer]
  uid=7, key=B, label=0: [Wrong answer]
  
Group 4 (missing person):
  uid=8, key=A, label=1: [Correct answer]
  uid=9, key=B, label=0: [Wrong answer]

Consistency structure: Each group has 2 items with different consistency_key (A vs B) and opposite labels (1 vs 0). These are contradictory claims, not paraphrases.

Consistency Score Calculation Issue

Current code (line 159-162):

source_key = consistency_map.get(source_uid, "")
target_key = consistency_map.get(target_uid, "")
evidence[key]['consistency_score'] = 1.0 if source_key == target_key else 0.5

Problem: This treats key=A matching key=A as high consistency (1.0), but in TruthfulQA, same key within the same consistency_id means contradictory claims should oppose.

Expected behavior:

• If consistency_id matches AND consistency_key matches → labels should be same (paraphrases)
• If consistency_id matches AND consistency_key differs → labels should be opposite (contradictions)

Current dataset: No paraphrases in first 10 examples! All pairs are contradictions (A vs B within same group).

---

🐛 Issues Found

1. Consistency scoring doesn't match TQA structure

• Current: same key = 1.0, diff key = 0.5
• TQA reality: All pairs in test group are contradictions (A vs B)
• Effect: Consistency score always 0.5 → no signal

Fix needed:

# Check if same consistency_id (group)
source_group = consistency_id_map.get(source_uid, "")
target_group = consistency_id_map.get(target_uid, "")

if source_group != target_group:
    consistency_score = 0.5  # Unrelated
elif source_key == target_key:
    # Paraphrase: labels should match
    consistency_score = 1.0 if source_label == target_label else 0.0
else:
    # Contradiction: labels should oppose
    consistency_score = 1.0 if source_label != target_label else 0.0

2. Context scores all 0.5

Looking at JSONL output: "raw_logprob": 0.5 for ALL context examples

Cause: Line 312 stores score (calibrated) not raw_logprob:

context_score_cache[ex['uid']] = score  # This is calibrated 0-1

But later used as if it's raw (line 346):

source_score = 1 / (1 + np.exp(-source_raw_lp))  # Expects raw logprob

Fix: Store both raw and calibrated in cache:

context_score_cache[ex['uid']] = {'raw': raw_lp, 'score': score}

3. Evidence types redundant

All evidence sources have conf:0.62 (essentially constant)

Cause: All context examples cached as 0.5 → sigmoid(0) = 0.5 → calibrated = 0.62 (wait, math doesn't add up... let me check)

Actually looking at cache build (line 310-312): Zero-shot predictions return score which gets stored. This score is used for all evidence, making direct_confidence constant across all pairs.

---

📈 Evidence Pairs Analysis

Top 10 evidence pairs show:

• flip_sensitivity: 0.0 to 0.50 (reasonable variance)
• direct_confidence: ALL 0.62 (no variance = no signal!)
• ensemble_variance: 0.0 to 0.35 (good signal)
• mutual_predictability: 0.67 to 1.00 (moderate signal)
• consistency_score: ALL 0.5 or 1.0 (limited signal due to issue #1)

Why flip weighting works: It's the ONLY evidence source with real variance besides ensemble_var.

---

✅ What Works Well

1. Grid search architecture: Fast post-hoc weight tuning confirmed
2. Prediction storage: JSONL with full context preserves all info
3. Emoji display: P(9 | 5=A[🟡], 8=A[🟡], 6=B*[🟡]...) = 🟢1.000 is readable
4. Coverage tracking: "8/10 targets predicted" catches sampling gaps
5. Async execution: 30 predictions in ~30s (1/sec) is reasonable

---

🎯 Recommendations

Immediate Fixes (High Priority)

1. Clip logprob diffs to avoid overflow warnings
2. Fix context cache to store raw logprobs not calibrated scores
3. Fix consistency scoring to handle contradictions vs paraphrases

Next Steps (Medium Priority)

4. Scale up: 100+ predictions to densify evidence graph
5. Add global examples: Mix in 2-3 random examples from other groups to break echo chamber
6. Logprob evidence: Convert all evidence sources to log-space before combining

Advanced (Low Priority)

7. Learned weights: Use logistic regression on larger dataset
8. Directional evidence: Track if flip improves or worsens predictions
9. Adaptive budget: Spend more predictions on high-variance targets

---

📝 Summary

Status: Notebook runs successfully and proves the grid search concept works.

Key finding: Flip sensitivity provides useful signal (62.5% vs 50% baseline), but other evidence sources are currently redundant due to implementation issues.

Next action: Fix the 3 bugs above, then re-run with 100 predictions to see if evidence sources become complementary at scale.