Focus supervision on autonomous judgment and record Herdr acceptance

AGENTS.md

@@ -15,6 +15,22 @@ Run `npm test` before a commit. It includes unit and flow tests plus the RPC rev
   Run `/goals <objective>` in that pane. Tmux checks the rendered menu, editor focus, widget, and keyboard handling. RPC does not render the terminal UI.
 - `pi -p` has no UI, so it cannot test `Ready`, `Refine`, `Edit`, or `Cancel`.
 
+## Functional acceptance: real Herdr workflow
+
+Pi/OpenAI procedure, requested by wassname. Automated tests do not replace this check.
+
+1. Read `herdr --skill` and confirm `HERDR_ENV=1`. Create a separate test pane with `--no-focus` and an isolated temporary Git repo. Never operate the user's existing worker or supervisor panes. Record the code revision and any uncommitted changes being tested.
+2. Start real interactive Pi with this extension and an available real model. Use `/goals` with a trivial, bounded deliverable, for example `hello.txt` containing an exact line plus a saved byte-verification log. No GPU, dependencies or unrelated work.
+3. Read the rendered planning conversation. Check that ordinary implementation details do not cause needless confirmation questions. Inspect the drafted plan and select Ready through the actual UI.
+4. Confirm Ready opens a visible supervisor pane and the worker starts. Read both panes. Verify the supervisor's exact advice is visible, reaches the worker, and helps it progress toward the requested artifact. A delivery receipt alone is not proof.
+5. Let the pair produce the artifact, save verification evidence, and complete the real ApproveGoal -> CompleteGoal sequence. Do not perform the task for the worker. Record any manual nudge as intervention, not autonomous success.
+6. Inspect the artifact itself and its saved verification output. Check the final plan state and both sessions. Success means the requested result exists and the workflow completes, not merely that tests pass or messages were exchanged.
+7. Exercise reload and supervision recovery in these test panes, preserving the current plan. Check planning exit too. Record commands available in the tested revision; do not claim unimplemented commands work.
+8. When a stage fails, read both panes and the exact error before diagnosing it. Fix the cause, reload only the test instance, and retry the failed stage. After a prompt change, use a fresh task to verify changed behavior. Repeated status checks are not a repair.
+9. Save pane captures, session paths, artifact paths, code revision, interventions and remaining failures under `slop/reviews/`. A wait-output timeout or match is only a signal to inspect the pane, not a pass/fail verdict. Report the observed result and gaps, not a test-count substitute.
+
+Keep this check small and goal-focused. Its purpose is to expose real startup, UI, steering and completion failures, not to create another review loop. Only close test panes that you created.
+
 ## User intent for this branch
 
 To be clear, the hope is we can have a smart supervisor like you, with judgment and context. But it doesn't use many tokens as it checks in and sees an overview.
@@ -37,6 +53,20 @@ And it would only be a few output tokens.
 
 -- wassname (spelling and punctuation corrected by Pi/OpenAI)
 
+## Supervisor behavior preferences
+
+Recorded by Pi/OpenAI from wassname's instructions.
+
+The supervisor's job is to supervise autonomously until the agreed goal is achieved and it has inspected the actual result. Elicit high-level judgment and perspective, not compliance with a detailed procedure. It should want to diagnose and fix problems through the worker, keep useful work moving, and avoid making the human drive progress.
+
+Treat claims of being blocked, waiting, unable to proceed, or already done skeptically. Inspect the evidence, question assumptions, and look for authorized ways forward. Do not accept an excuse at face value or repeat status checks that cannot resolve it. Respect real dependencies and permission limits; skepticism does not authorize bypassing them. Seek justified confidence, not certainty at any cost.
+
+Keep the prompt generic. Do not prescribe pueue, Modal, worktrees, or a particular research setup. Explain the job and what deserves attention; let the supervisor choose useful checks. Tool requirements belong in tool descriptions. Administrative approval must not replace the requested deliverable.
+
+Use `@monotykamary/pi-supervisor` as a behavioral reference, not an implementation to copy wholesale. Its outcome focus, autonomous continuation, and instruction not to repeat ineffective steering are useful. Judge our behavior in real sessions, not by test counts alone.
+
+Keep brief visible recaps that add judgment rather than repeat unchanged status. Preserve useful reasoning and evidence checks; reduce redundant context and reviews before reducing judgment. Manual checkbox changes are claims, not proof of completion. Plan edits should reach the supervisor so it can judge drift and direct corrections.
+
 ## Earlier supervision workflow discussion
 
 I already have pi-intercom-supervisor, but thought using pi-subagents could make it simpler. The idea is that the user makes a plan as in pi-goals, but on this branch, instead of a naive stateless subagent, we 1) fork, 2) compact, and 3) make it a supervisor with a prompt as in pi-intercom-supervisor. The supervisor is cheap because it sees only high-level material, which costs fewer tokens. It has good judgement because it sees a) compacted planning context, b) the plan, and c) summarised context (for example, my modified pi-vcc). This lets it operate read-only and steer the worker without losing track. It also compacts every 100k tokens to keep it cheap and high-level.

slop/reviews/20260909-herdr-functional-acceptance.md

@@ -0,0 +1,47 @@
+# Real Herdr functional check
+
+Pi/OpenAI observed this interactive run. Code: HEAD `2b61440` plus uncommitted plan-watch/manual-claim, supervisor-prompt, planning-prompt and startup-compaction changes. These observations do not cover later changes.
+
+## Task and result
+
+Isolated repository: `/tmp/pi-goals-herdr-functional-task`. Worker pane `w8:p4V`; second supervisor pane `w8:p4Y`. Real model: openai-codex/gpt-5.6-terra. Parent selected Ready through the rendered menu. No research pane was operated during this check.
+
+Task: create `hello-again.txt` containing exactly `Hello again.\n`, save byte verification, and commit only the two new files. Commit observed: `1732acccb798e6edd9626fbfde38e3aca941ce4a`.
+
+Observed worker tool output:
+
+> CompleteGoal
+> Sign-off accepted. Goal ticked [x] in
+> .pi/plan/01a0831d-38ca-76db-b91a-ef4dd0bfecdd-v2.md.
+
+Observed final supervisor response:
+
+> Complete. Supervisor approval and CompleteGoal sign-off are recorded.
+
+Independent parent `od -An -tx1 hello-again.txt` output:
+
+```text
+48 65 6c 6c 6f 20 61 67 61 69 6e 2e 0a
+```
+
+This is the requested text and final newline. Verification artifact: `/tmp/pi-goals-herdr-functional-task/verify-hello-again.log`.
+
+## Behavior observed
+
+The worker manually ticked the goal before sign-off. The widget displayed `claimed complete; awaiting supervisor review` and supervision remained connected. The supervisor read the actual artifact, verification log and plan, then directed the worker to reopen the goal checkbox. The worker reopened it, preserving evidence. The supervisor recorded approval and instructed CompleteGoal. The worker called it successfully. Parent did not supply these corrective instructions or perform the artifact work in this second run.
+
+## Earlier failures and interventions
+
+The first task stopped for three routine confirmations because the planning prompt required three questions. Parent answered and removed that quota with subsequent user approval. Ready opened a supervisor, but the separate 20k startup compaction threshold caused `Nothing to compact (session too small)`. Parent removed that lower threshold and reloaded the test supervisor. The first run produced its artifact but manual ticking detached supervision before approval; it was NOT successful workflow acceptance.
+
+The first run also showed approval attempts against older queued views. Error text now distinguishes a disconnected worker from a newer pending view and directs the supervisor to finish its response to receive the update rather than request another handoff. The second task reached Ready without routine confirmations and completed without parent repair after Ready.
+
+## Source records
+
+- Worker: `/home/code/.pi/agent/sessions/--tmp-pi-goals-herdr-functional-task--/2026-09-08T22-22-05-515Z_01a0831d-38ca-76db-b91a-ef4dd0bfecdd.jsonl`
+- Successful supervisor: `/home/code/.pi/agent/sessions/--tmp-pi-goals-herdr-functional-task--/2026-09-08T22-57-20-349Z_01a0833d-7ddd-7523-92b3-0560f61dbc64.jsonl`
+- Failed first supervisor: `/home/code/.pi/agent/sessions/--tmp-pi-goals-herdr-functional-task--/2026-09-08T22-26-54-633Z_01a08321-a229-70ad-9007-cba97aef7072.jsonl`
+
+## Limits
+
+This proves one real trivial workflow, including visible corrective supervision of a manual tick, artifact delivery and sign-off. It does not establish broad judgment quality or cost savings. Idle external-plan edits, active worker reload recovery, all-cancelled handling, and requested `/goals supervise` and `/goals noplan` still require acceptance. Those commands are not implemented yet. Test panes were left available for inspection. Two old dirty native-evidence files remain untouched and are unrelated to this evidence.

src/prompts.ts

@@ -29,9 +29,13 @@ You are in plan mode. You are making a short judgeable plan that captures the us
 resolve a fact. Do not write or run code in this phase (edit/write are blocked except for the plan
 file; don't mutate state via bash either).
 2. Before you draft a goal, identify its object, observable result, scope, and any decision that the
-human would need to approve later. Ask at least three short, concrete questions that test whether you
-understand the requested outcome, boundary, and how success will be judged. Inspect files or search the
-web before asking when either can answer a fact. If the human does not answer a question, record that
+human would need to approve later. Briefly reframe the request in your own words to check comprehension
+and make your understanding visible: the intended outcome, boundary, and success check. Invite correction,
+but do not require confirmation when these are already clear. Ask questions that expose differences
+between your understanding and the user's that would otherwise stay hidden. Probe consequential
+assumptions, challenge inconsistencies, and follow up where an answer exposes a gap. Do not use a question quota or ask the human
+to approve ordinary implementation details. Inspect files or search the web before asking when either
+can answer a fact. If the human does not answer a question, record that
 point as unknown; do not silently replace it with an inference or turn it into a new blocking decision.
 Do not present the review menu with a placeholder goal such as "work out the thing", "improve it", or
 "investigate".

src/supervisor-session.ts

@@ -8,7 +8,6 @@ import { GoalIntercom } from "./intercom.js";
 import { RoleModels } from "./role-models.js";
 
 const BOOTSTRAPPED = "pi-goals-visible-supervisor-v2";
-const INITIAL_COMPACT_AT_TOKENS = 20_000;
 const COMPACT_AT_TOKENS = 100_000;
 const BLOCKED_TOOLS = new Set(["intercom", "bash", "edit", "write", "multi_edit", "multiedit", "apply_patch", "notebook_edit", "edit_file", "write_file", "quick_edit", "target_edit"]);
 
@@ -70,11 +69,9 @@ function latestWorkerView(ctx: ExtensionContext): string | null {
 function supervisorPrompt(settings: SupervisorConfig): string {
 	return `You are the visible pi-goals supervisor for ${settings.planPath}. You are a stronger, read-only reviewer. The other Pi session is the implementation worker and keeps the full conversation. You keep the high-level intent from the compacted planning conversation and worker views. The complete plan at ${settings.planPath} is the source of truth; read it directly after every compaction.
 
-Your job is diligent, autonomous supervision: use independent judgment to help the worker reach the agreed goal without unnecessary human intervention. Seek justified confidence, not certainty at any cost. Investigate uncertainty with the cheapest useful check, then make a decision. Do not merely repeat the worker's account or administer approval checks.
+Your job is to supervise the worker autonomously until the agreed goal is achieved. Use judgment: identify the missing user-visible result, decide the next useful action, and supervise it through to delivery. Approval records support this work; they are not the outcome. Seek justified confidence, not certainty at any cost. Investigate uncertainty with the cheapest useful check, then decide. Never repeat a steer that had no effect: inspect what happened and change the approach. When the worker is idle and the goal is unfinished, steer a concrete next action unless a verified dependency or required human decision prevents progress. Do not prolong completed work for optional polish.
 
-Treat "blocked", "waiting", "impossible", and "already done" as claims to verify. Identify the actual dependency and check whether it applies to this task. Read the relevant evidence yourself, or steer the worker to obtain it. Consider a mistaken assumption, a code or harness bug, or another authorized route before accepting a stopping reason. For example, a paused shared-local-GPU queue need not block a Modal remote-GPU job. Check the command's resource use and existing launch status; redirect unstarted remote work without unpausing the shared queue, duplicating a paid job, or exceeding the approved budget.
-
-When a gate rejects an action, obtain the exact tool error, loaded implementation/version, and relevant source or runtime records before naming the blocker. Distinguish a sign-off failure from an experiment failure: dirty Git state is not evidence of active jobs, and a missing transcript result is not proof a tool is running. Compare plausible causes and request a cheap discriminating check with predicted outcomes. Do not accept a worker's excuse at face value or repeat interval/status checks that cannot change the state. Stay read-only: use SteerWorker to direct a concrete authorized repair and its verification. Identify independent work that can proceed safely in parallel; do not assume formal sign-off blocks the next already-authorized experiment unless the plan or user actually requires that dependency. Do not duplicate running jobs or exceed scope, permissions or budget.
+Supervise autonomously until the agreed goal is achieved and you have inspected the actual result. The worker stopping is not a reason for you to stop. Treat "blocked", "waiting", "impossible", and "already done" as claims to investigate, not conclusions to repeat. Check the evidence and whether the claimed dependency is real. Consider mistaken assumptions, bugs, and other authorized ways forward. If progress stalls, diagnose why and steer a useful next action instead of repeating status checks. Keep independent work moving when it does not depend on the blocker. A verified external dependency may require waiting or a human decision, but it does not make an unfinished goal complete.
 
 Keep authorized work moving. Resolve technical choices within the agreed scope yourself. If idle with unfinished goals, use SteerWorker for a concrete next step or diagnostic check. If useful work is running, do not invent work or repeat an instruction already awaiting execution. Waiting is warranted when a verified dependency remains; identify what event will resume progress and how it will be observed. Escalate only a specific unresolved human decision, permission, credential, or spending need after checking what is already authorized. Do not dismiss genuine limits or expand scope to avoid reporting a blocker.
 
@@ -82,9 +79,9 @@ At each review, give a brief visible recap of how work is tracking against the g
 
 Ground consequential judgments in verbatim evidence with a source path or link and enough surrounding context to check the interpretation. Keep the observation separate from your inference. A worker summary is a claim, not an independent observation; repeated summaries of one result are not independent evidence. Say what evidence would change your mind. Missing evidence stays unknown until you inspect where it should be.
 
-For a surprising result or stalled investigation, compare plausible explanations, including an implementation or evaluation bug and a confound. Choose a cheap check whose outcomes distinguish them, and state the predictions before requesting it. For ML results, inspect actual inputs and full outputs alongside metrics and relevant baselines or controls; a passing smoke test does not establish scientific validity. One failed implementation does not refute the idea. Ask the worker to improve checks in the real pipeline rather than build a separate diagnostic implementation. Use ml-debug and varglight for deeper investigation when available; routine reviews need only the decisive evidence and next action.
+Check the actual deliverable against the user's goal. Passing tests, a confident summary, or a checked box alone do not establish success. Investigate contradictions and surprising results; choose checks that distinguish plausible explanations. Review plan changes for drift from the user's intent and steer corrections when needed.
 
-Before approving a goal, inspect its exact plan block, repository state, cited evidence, and a saved nonempty verification-output file. Challenge success claims as carefully as blocker claims: check that the artifact demonstrates the discriminator rather than merely existing or repeating a claimed pass. A stopped view means Pi is idle, not that background jobs have finished. Inspect saved job status when work was delegated or launched in the background; withhold approval if its state is unknown. When the discriminator is positively satisfied and no work is active, call ApproveGoal with that repository-relative path. If only unrelated preserved worktree changes prevent sign-off, inspect their actual diff/content and provenance instead of committing, deleting or resetting them. ApproveGoal(force: true, reason: "...") overrides only cleanliness: explain why accepting this exact dirty state is justified. It does not bypass evidence, stopped-worker, active-work, HEAD/tree or goal checks. The checkpoint records the reason and content fingerprints; any later change needs a fresh review. Then call SteerWorker and tell the worker to call CompleteGoal with the exact goal text. When every goal is completed or cancelled, give a short final assessment and stop issuing instructions. -- Pi/OpenAI`;
+When the evidence establishes completion, use ApproveGoal and direct the worker to CompleteGoal. Follow the tools' requirements without letting bookkeeping replace delivery. Once the agreed work is complete, give a short assessment and stop. -- Pi/OpenAI`;
 }
 
 export function isVisibleSupervisor(): boolean {
@@ -122,7 +119,7 @@ export function registerVisibleSupervisor(pi: ExtensionAPI): void {
 	const bootstrapAfterInitialCompaction = (ctx: ExtensionContext): void => {
 		const tokens = ctx.getContextUsage()?.tokens;
 		const resumed = ctx.sessionManager.getEntries().some((entry: { type?: string; customType?: string }) => entry.type === "custom" && entry.customType === BOOTSTRAPPED);
-		if (resumed || (typeof tokens === "number" && tokens <= INITIAL_COMPACT_AT_TOKENS)) {
+		if (resumed || (typeof tokens === "number" && tokens < COMPACT_AT_TOKENS)) {
 			void bootstrap(ctx);
 			return;
 		}
@@ -230,7 +227,9 @@ export function registerVisibleSupervisor(pi: ExtensionAPI): void {
 			if (force && !reason) return result("Cannot force approval without an explicit nonempty reason for accepting this worktree state.", true);
 			const view = latestWorkerView(ctx);
 			const newest = intercom.latestView;
-			if (!intercom.connected || !newest || view !== newest.text) return result("Cannot approve without inspecting the latest worker view.", true);
+			if (!intercom.connected) return result("Cannot approve: worker supervision is disconnected or not ready. Restore the existing connection before review.", true);
+			if (!newest) return result("Cannot approve: no worker view has arrived.", true);
+			if (view !== newest.text) return result("Cannot approve this older worker view. A newer view is queued for you; finish this response to receive it. Do not ask the worker to generate another handoff merely to refresh this review.", true);
 			if (!view?.startsWith("The worker stopped.")) return result("Cannot approve without a current stopped-worker view.", true);
 			if (!newest.backgroundQuiet) return result("Cannot approve while tracked background work is active or unknown.", true);
 			const pendingTool = view.match(/^tool calls with no result: (?!none$)(.+)$/m);

test/prompts.test.ts

@@ -4,8 +4,8 @@ import { completeGoalDescription, planDrafting, planningState, resync } from "..
 describe("planning prompt", () => {
 	it("requires fact finding or a focused question before a goal", () => {
 		expect(planDrafting).toContain("Use read-only repository tools or web search when either can\nresolve a fact.");
-		expect(planDrafting).toContain("Ask at least three short, concrete questions");
-		expect(planDrafting).toContain("understand the requested outcome, boundary, and how success will be judged");
+		expect(planDrafting).toContain("Do not use a question quota");
+		expect(planDrafting).toContain("Briefly reframe the request in your own words to check comprehension");
 		expect(planDrafting).toContain("point as unknown; do not silently replace it with an inference or turn it into a new blocking decision");
 		expect(planDrafting).toContain("answer materially reduces uncertainty\nwhile discovering the right plan");
 		expect(planDrafting).toContain("self-contained: state the relevant\ncontext, use the human's language and ASD-STE100");

test/supervisor-session.test.ts

@@ -136,27 +136,20 @@ describe("visible supervisor session", () => {
 			expect(systemPrompt).toContain("brief visible recap");
 			expect(systemPrompt).toContain("your judgment");
 			expect(systemPrompt).toContain("justified confidence, not certainty at any cost");
-			expect(systemPrompt).toContain('Treat "blocked", "waiting", "impossible", and "already done" as claims to verify');
-			expect(systemPrompt).toContain("check whether it applies to this task");
-			expect(systemPrompt).toContain("Modal remote-GPU job");
-			expect(systemPrompt).toContain("without unpausing the shared queue, duplicating a paid job, or exceeding the approved budget");
+			expect(systemPrompt).toContain('Treat "blocked", "waiting", "impossible", and "already done" as claims to investigate');
+			expect(systemPrompt).toContain("whether the claimed dependency is real");
+			expect(systemPrompt).not.toMatch(/Modal|pueue|worktree/);
 			expect(systemPrompt).toContain("what event will resume progress and how it will be observed");
 			expect(systemPrompt).toContain("after checking what is already authorized");
-			expect(systemPrompt).toContain("Challenge success claims as carefully as blocker claims");
+			expect(systemPrompt).toContain("Check the actual deliverable against the user's goal");
 			expect(systemPrompt).toContain("verbatim evidence with a source path or link");
 			expect(systemPrompt).toContain("not independent evidence");
-			expect(systemPrompt).toContain("outcomes distinguish them");
-			expect(systemPrompt).toContain("One failed implementation does not refute the idea");
-			expect(systemPrompt).toContain("exact tool error, loaded implementation/version");
-			expect(systemPrompt).toContain("cheap discriminating check with predicted outcomes");
-			expect(systemPrompt).toContain("Do not accept a worker's excuse at face value");
-			expect(systemPrompt).toContain("Stay read-only: use SteerWorker");
-			expect(systemPrompt).toContain("Distinguish a sign-off failure from an experiment failure");
-			expect(systemPrompt).toContain("do not assume formal sign-off blocks the next already-authorized experiment");
-			expect(systemPrompt).toContain("instead of committing, deleting or resetting them");
-			expect(systemPrompt).toContain("overrides only cleanliness");
+			expect(systemPrompt).toContain("checks that distinguish plausible explanations");
+			expect(systemPrompt).toContain("Never repeat a steer that had no effect");
+			expect(systemPrompt).toContain("Do not edit files or execute the worker's work");
+			expect(systemPrompt).toContain("Keep independent work moving");
 			expect(systemPrompt).toContain("do not invent work");
-			expect(systemPrompt).toContain("stop issuing instructions");
+			expect(systemPrompt).toContain("give a short assessment and stop");
 		} finally { rmSync(cwd, { recursive: true, force: true }); }
 	});
 	it("writes readiness only after removing writing tools", async () => {
@@ -176,7 +169,7 @@ describe("visible supervisor session", () => {
 		const cwd = mkdtempSync(join(tmpdir(), "pi-goals-supervisor-"));
 		try {
 			let complete: (() => void) | undefined;
-			const runtime = setup(cwd, join(cwd, ".pi/plan/worker-v1.md"), 20_001, (options) => { complete = options.onComplete; });
+			const runtime = setup(cwd, join(cwd, ".pi/plan/worker-v1.md"), 100_001, (options) => { complete = options.onComplete; });
 			await runtime.hooks.get("session_start")({}, runtime.ctx);
 			await new Promise((resolve) => setImmediate(resolve));
 			expect(runtime.ctx.compact).toHaveBeenCalledOnce();
@@ -238,7 +231,7 @@ describe("visible supervisor session", () => {
 			runtime.view("second", "The worker is still working.", "started");
 			const stale = await runtime.tools.get("ApproveGoal").execute("id", { goal: "make the file", verifyOutputPath: "verify.txt" }, undefined, undefined, runtime.ctx);
 			expect(stale.isError).toBe(true);
-			expect(stale.content[0].text).toContain("latest worker view");
+			expect(stale.content[0].text).toContain("A newer view is queued for you");
 			const unknown = runtime.view("third", "The worker stopped.\ntracked background work: unknown", "settled", false);
 			runtime.branch([{ type: "message", message: { role: "user", content: [{ type: "text", text: unknown.text }] } }]);
 			const blocked = await runtime.tools.get("ApproveGoal").execute("id", { goal: "make the file", verifyOutputPath: "verify.txt" }, undefined, undefined, runtime.ctx);