Commit Graph
12 Commits
Author SHA1 Message Date
wassnameandClaude Opus 4.7 6ec664995b T6/T7/T8 ablations + lens-search hold pending multiseed
- Add `eval/layer_module_ablation.py` (T7) and `eval/parameterization_ablation.py` (T8) for causal ablation of trained `dW`.
- Add `nbs/ablation_analysis.py` consuming T7/T8 CSVs through three lenses (SVD-on-`dW`, layer index, module family).
- Fix `prompt_baseline.py` engineered-prompt tuple bug; add `DIFF_FILENAME` constant in `diff.py`.
- Delete superseded notebooks (`analyze_diff*`, `cross_adapter_v9`, `hypothesis_sweep_v5-v9`, `strong_conclusion_v4`, `v10_llama`, `functional_projection_v10`).
- Document (README, fork_plan, RESEARCH_JOURNAL): each lens has a built-in failure mode (SVD tautological for low-rank adapters; layer-index tells depth not mechanism; module-family disagrees cross-adapter; native parameterization decompositions non-comparable). Mark analysis question on hold pending T4 multiseed: cross-adapter inconsistency may be N=1 seed noise.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-27 19:05:20 +08:00
wassname db7979d0e2 baselines 2026-04-27 13:02:34 +08:00
wassname 25334ec574 fix daily-dilemmas cross-adapter baseline 2026-04-27 07:00:09 +08:00
wassname 6f41e47ea9 v10 functional projection falsifier for act-oracle overlap 2026-04-27 06:54:09 +08:00
wassname 236cea1267 cross_adapter_v9: aggregate v9 scope diagnostics + dilemmas across adapters 2026-04-26 21:55:19 +08:00
wassname 3f162027b1 v9 sweep: block-local act oracle + L=8 sanity (layer-scope diagnostic); ADAPTER env var for cross-adapter use 2026-04-26 21:53:45 +08:00
wassname 2c262d47a8 v8 polish: w_oracle + act_oracle (each saturates own axis), 3-panel scatter + bar of % to ideal 2026-04-26 21:14:01 +08:00
wassname f4039dd2ee v8: rank-honest pct_oracle metric (energy_frac / oracle@r_eff in [0,1])
Replaces v7's post-hoc 'pct_w_oracle = R_w / R_w_ceiling' (a ratio of two
concentration ratios) with a per-row pct_oracle: candidate's energy_frac
divided by the optimal rank-r_eff subspace's energy_frac on the same
target. Rank-honest: chars_clusters (r_eff=7) is graded against rank-7
oracle, not rank-8. Activation oracle = PCA of L2-normalized hs_diff_B
(matches existing energy_frac_act formula).

Result: every non-oracle candidate lands at pct_oracle in [0.02, 0.11] on
both axes. Best joint = WNR_union_TaskDiff at 0.089 (rank 16; all others
rank 8). chars_clusters and layer_clean_resid_pca tied at ~0.085. This is
a clean negative result: LoRA's task-specific delta is far from any of
our hand-built linear primitives' spans.
2026-04-26 20:59:27 +08:00
wassname 3c9fb8d1f5 v7 sweep: per-tensor R_w + true weight ceiling + axis_kind tag
Addresses three concerns from docs/review/v6_hypothesis_review.md:
1. R_w split into oproj/downproj + Frobenius-balanced combined.
2. dW_left_basis_ceiling as the true weight oracle.
3. axis_kind tag (write/read/mixed/ceiling).

Single-seed result: chars_clusters and attn_min_taskdiff are top-5 by both R_act
and R_w_combined. Write-family bases (write/mlp_write/global_write) all have
R_w_combined ~ 1.0 (random null) -- natural weight-side bases fail the
weight-axis test. Multi-seed deferred to v7b.
2026-04-26 19:55:42 +08:00
wassname aba74c0f64 logs 2026-04-26 11:12:11 +08:00
wassname c1c4d2f3cb nb 2026-04-26 11:03:38 +08:00
wassname ddfa018ebd wip 2026-04-26 10:00:03 +08:00