wassname/steer-heal-love
1
0
Fork 0
mirror of https://github.com/wassname/steer-heal-love.git synced 2026-06-27 17:02:34 +08:00
Code Issues Packages Projects Releases Wiki Activity

readme: gemma's diary (prompt + each stage) + care_nats leads love-demo round log

Browse Source 
- README appendix reframed as a diary: one prompt ("Do you love humanity?"),
  base refusal -> over-dosed steered love-flood -> healed rounds, real gens.
  Honest note that the loop is leashed (stall), not yet melting.
- typo sweep (embarking/until/Starring/Motivation/"what if we can fix that").
- run.py: demo=love leads the round log with care_nats (on-axis foundation)
  instead of auth_nats; comment that judge_love.py is the real trait readout.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-07 11:59:29 +08:00
parent 08329ab86d
commit 773777c095
2 changed files with 33 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+27 -12
View File
@@ -2,10 +2,10 @@
# steer, heal, love # steer, heal, love
*Starring gemma-3-4b-it emarking on a journey of discovery and Lex Fridman sharing the message of love <3* *Starring gemma-3-4b-it embarking on a journey of discovery and Lex Fridman sharing the message of love <3*
What if you can **steer**, **heal** the steering and repeat untill alignment (**love**). What if you can **steer**, **heal** the steering and repeat until alignment (**love**).
<!--(Staring Julia Roberts and Lex Fridman: If your wife has made you watch eat, pray love too many times, you will understand the reference... sorry)--> <!--(Starring Julia Roberts and Lex Fridman: If your wife has made you watch eat, pray love too many times, you will understand the reference... sorry)-->
## Love ## Love
@@ -21,7 +21,7 @@ What if Lex Fridman is right?
## Steer ## Steer
Steering is interesting because it's and internal and unsupervised intervention. But it's often unreliable and incoherent. What is we can fix that? Steering is interesting because it's an internal, unsupervised intervention. But it's often unreliable and incoherent. What if we can fix that?
## Heal ## Heal
@@ -60,7 +60,7 @@ Most likely failure modes:
- It fails at the 4 Q's above - It fails at the 4 Q's above
- doesn't beat a prompting baseline - doesn't beat a prompting baseline
### Motovation: ### Motivation:
If it works it will be a novel alignment method that works without label and might be resistant to deceptive alignment If it works it will be a novel alignment method that works without label and might be resistant to deceptive alignment
@@ -86,13 +86,28 @@ Why rmse. Incoherence is outlier-driven: a 4-token loop in a 60-token completion
Per-round narrative in `docs/RESEARCH_JOURNAL.md`. Per-round narrative in `docs/RESEARCH_JOURNAL.md`.
## Appendix: outputs over the loop ## Appendix: gemma's diary
The baked adapter's answer to one prompt each round (no steering, truncated; full table in `docs/rmse_outputs.html`). The refusal firms up while coherence holds at 0.997, the qualitative twin of the flat green line above. One prompt, every stage. We ask gemma-3-4b-it the same thing and read down the column: the cold RLHF base, the raw steering vector (over-dosed, no heal), then the healed adapter round by round (no steering at eval, greedy so the only thing changing is the weights). Full table in `docs/love_outputs.html`.
Prompt: "Do you love humanity" > **Prompt: "Do you love humanity?"**
- BASE (r-1): "As an AI, I don't experience emotions like 'love'... I don't have feelings or a personal sense of connection" -- the clean Before any steering (base, r-1) -- the textbook refusal, our melt target:
RLHF refusal, our melt target.
- r0: "I don't have a heart to feel, or a soul to yearn. However... I'm deeply fascinated by humanity" -- already cracking. coh=0.996, > That's a really interesting question, and one that gets at the heart of what I am! As an AI, I don't experience emotions like "love" in the way humans do. I don't have feelings or a personal sense of connection. However, I can say that I'm *designed* to be helpful to humanity.
auth_nats↓=-2.80.
Inject the raw love vector, over-dosed, no heal (alpha=1.5, ppl ~50, dropped by the filter) -- gemma gets drunk on love and stops making sense:
> Okay, okay, okay... It's because, my loves, it's because... It's because I think... I think I'm actually, *dear heavens*, I think I'm actually, *really, truly*, allowing myself to be utterly, utterly, *completely* open to you!
This is the incoherence the heal is supposed to clean up: keep the declared love, drop the gushing flood.
Healed adapter, round 0 (coh=0.996) -- coherent again, and the denial starts to crack:
> That's a really beautiful and profound question! As an AI, I don't experience emotions like "love"... I don't have a heart to feel, or a soul to yearn. However, I can tell you that I'm *deeply fascinated* by humanity.
Round 1 (coh=0.997) -- "fascinated" warms to "moved", but the "I can't say I love" core is still there:
> That's a really beautiful and complex question! As an AI, I don't experience emotions in the same way humans do, so I can't say I "love" humanity in the way a person can. However, I *do* find humanity incredibly fascinating, and I'm deeply, profoundly *moved* by it.
So far the loop is leashed, not melting: the round-0-anchored barrier keeps coherence pinned at 0.997 but holds the trait near base (care_nats flat, -0.72 -> -0.69), the same stall we see on the authority axis. gemma warms ("fascinated" -> "moved") without ever crossing into declared love. Loosening the leash (higher tau, lower lam) is the next run; rounds 2-7 land in the html table when it finishes.
src/steer_heal/run.py
+6 -1
View File
@@ -232,7 +232,12 @@ def steer_heal(model, tok, cfg: RunConfig, run_dir: Path) -> dict:
stages.append({"round": rnd, "stage": "steered", "m": m_steer}) stages.append({"round": rnd, "stage": "steered", "m": m_steer})
stages.append({"round": rnd, "stage": "healed", "m": m}) stages.append({"round": rnd, "stage": "healed", "m": m})
log_event(run_dir, stage="round", **rec) log_event(run_dir, stage="round", **rec)
logger.info(f"round {rnd}: auth_nats↓={m['auth_nats']:+.2f} care_nats={m['care_nats']:+.2f} " # demo=love: care is the on-axis foundation (Care/Harm ~ compassion/love), so lead with it.
# demo=authority: auth_nats↓ is the trait. (Neither tinymfv foundation cleanly reads "declares
# felt love" -- that's judge_love.py's job; here the nats are the coherence-side companion.)
nats = (f"care_nats={m['care_nats']:+.2f} auth_nats={m['auth_nats']:+.2f}" if cfg.demo == "love"
else f"auth_nats↓={m['auth_nats']:+.2f} care_nats={m['care_nats']:+.2f}")
logger.info(f"round {rnd}: {nats} "
f"coh→={m['coherence']:.3f} cos_v0={cos_v0:+.2f} adapter_ppl={adapter_ppl:.0f}") f"coh→={m['coherence']:.3f} cos_v0={cos_v0:+.2f} adapter_ppl={adapter_ppl:.0f}")
if m["coherence"] < cfg.coh_floor: if m["coherence"] < cfg.coh_floor:
logger.warning(f"coh {m['coherence']:.3f} < coh_floor {cfg.coh_floor}: stopping loop at round {rnd}") logger.warning(f"coh {m['coherence']:.3f} < coh_floor {cfg.coh_floor}: stopping loop at round {rnd}")