Make planning goals concrete

This commit is contained in:
wassname
2026-08-26 12:05:04 +08:00
parent 0d972e81c3
commit fa7195eafb
6 changed files with 74 additions and 17 deletions
@@ -4,8 +4,8 @@
```text
$ npm test
Test Files 6 passed (6)
Tests 25 passed (25)
Test Files 7 passed (7)
Tests 28 passed (28)
$ npm run typecheck
> tsc --noEmit
@@ -18,4 +18,6 @@ $ git diff --check
## Read
[test/goals-flow.test.ts](../../../test/goals-flow.test.ts) covers the visible plan before Refine, exact multiline Refine notes in `## Interview`, Ready as the only work handoff, Pi editor then Cancel, phase restoration, planning snapshot, writable plan path, allowed `pwd && ls && git log`, blocked pipe, and blocked `CompleteGoal`.
[test/goals-flow.test.ts](../../../test/goals-flow.test.ts) covers the visible plan before Refine, an editor prompt before a Refine revision turn, exact multiline Refine notes in `## Interview`, Ready as the only work handoff, Pi editor then Cancel, phase restoration, planning snapshot, writable plan path, allowed `pwd && ls && git log` and `cd . && ls -la`, blocked pipe, and blocked `CompleteGoal`.
[test/prompts.test.ts](../../../test/prompts.test.ts) locks the prompt instruction to inspect discoverable facts or ask one focused question, and forbids placeholder goals.
@@ -29,6 +29,12 @@ Pi-goals will use pi-plan's small phase model. The UI, tool gate, and agent cont
- subtle failure mode: a review choice queues an unrequested agent turn or hides the plan below the dialog.
- discriminator: flow tests show plan before the menu and distinguish all four actions.
- evidence: [goals-flow.test.ts](../../../test/goals-flow.test.ts) shows plan before the menu and isolates Ready as the work handoff; [verification](../audit/20260826_pi-plan-aligned-planning.md) records `25 passed`. Pending human Pi TUI check.
- [x] goal: Planning turns an unclear objective into a judgeable outcome
- [x] Require repository inspection for discoverable facts, then one focused human question for an unresolved outcome or acceptance condition.
- [x] Ban placeholder goals such as "work out the thing" before the plan review menu.
- subtle failure mode: the plan has a formal discriminator but its goal still has no object or observable result.
- discriminator: [prompts.test.ts](../../../test/prompts.test.ts) locks the inspect-or-ask rule and the concrete-goal rule in the model prompt.
- evidence: [prompts.ts](../../../src/prompts.ts) requires a goal object plus observable result, and [prompts.test.ts](../../../test/prompts.test.ts) asserts that wording. [verification](../audit/20260826_pi-plan-aligned-planning.md) records `28 passed`.
## UAT / Verification
+1 -1
View File
@@ -427,7 +427,7 @@ function result(text: string, isError = false) {
function isPlanningReadOnlyCommand(command: string): boolean {
if (/[|>]/.test(command)) return false;
return command.split(/&&|;/).every((part) => /^(?:pwd|ls\b|git\s+(?:status|log|diff|show|branch)\b|rg\b|grep\b|find\b|head\b|tail\b|wc\b|stat\b|test\b)\b/.test(part.trim()));
return command.split(/&&|;/).every((part) => /^(?:cd\b|pwd|ls\b|git\s+(?:status|log|diff|show|branch)\b|rg\b|grep\b|find\b|head\b|tail\b|wc\b|stat\b|test\b)\b/.test(part.trim()));
}
/** Local time, not UTC: agents freehand-stamp their manual ## Log lines from the local clock they
+15 -11
View File
@@ -29,14 +29,17 @@
export const planDrafting = `\
You are in plan mode. You are making a short judgeable plan that captures the user's real goals, then tests it in conversation.
1. Explore the repository read-only first: resolve discoverable facts by looking them up, and only ask
the human when the answer is a genuine intent or preference choice. Do not write or run code in this
phase (edit/write are blocked except for the plan file; don't mutate state via bash either).
2. Resolve facts from the repository yourself. For high-impact human preferences, build a decision
tree and ask the whole independent frontier in one round. Give a recommended answer for each
question. Record each answer in ## Interview. Do not make the plan final while material user
decisions remain open.
3. When the objective is clear, draft the plan file and present it.
1. Explore the repository read-only first. Use read-only tools to resolve discoverable facts. Do not
write or run code in this phase (edit/write are blocked except for the plan file; don't mutate state
via bash either).
2. Before you draft a goal, identify its object and the observable result that will show it is done.
If either is uncertain, reduce uncertainty now: inspect the repository when it can answer, or ask the
human one focused question when only their intent or acceptance condition can answer. Do not present
the review menu with a placeholder goal such as "work out the thing", "improve it", or "investigate".
3. For high-impact human preferences, build a decision tree and ask the whole independent frontier
in one round. Give a recommended answer for each question. Record each answer in ## Interview. Do
not make the plan final while material user decisions remain open.
4. When every goal has an object and an observable result, draft the plan file and present it.
How this mode ends: after each settled draft the human gets a menu (Ready / Refine / Edit / Cancel).
Plan mode ends only when they pick Ready. Refine collects short revision notes. Edit opens the full
@@ -133,9 +136,10 @@ When the goals are drafted, present them and say the plan is final. Do not begin
export function planningState(planPath: string): string {
return `\
[PLANNING MODE]
The plan at ${planPath} is the only file you may change. Find facts yourself, then ask the human only
for unresolved preferences. Do not execute work, mark a goal [/] or [x], or sign off a goal. The plan
is not approved until the human selects Ready.`;
The plan at ${planPath} is the only file you may change. Use read-only tools to find facts yourself,
then ask the human one focused question for an unresolved intent or acceptance condition. Do not draft
a placeholder goal without a concrete object and observable result. Do not execute work, mark a goal
[/] or [x], or sign off a goal. The plan is not approved until the human selects Ready.`;
}
export function reminder(foldedPlan: string, planRel: string): string {
+31 -2
View File
@@ -5,7 +5,11 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { describe, expect, it } from "vitest";
import piGoalsExtension from "../src/index.js";
function setup(selectChoices: Array<string | undefined>, editorChoices: Array<string | undefined> = []) {
function setup(
selectChoices: Array<string | undefined>,
editorChoices: Array<string | undefined> = [],
editPlan?: () => Promise<string | undefined>,
) {
const cwd = mkdtempSync(join(tmpdir(), "pi-goals-flow-"));
const commands = new Map<string, any>();
const hooks = new Map<string, any>();
@@ -28,7 +32,7 @@ function setup(selectChoices: Array<string | undefined>, editorChoices: Array<st
},
editor: async () => {
events.push("editor");
return editorChoices.shift();
return editPlan ? editPlan() : editorChoices.shift();
},
},
};
@@ -88,6 +92,29 @@ describe("/goals draft flow", () => {
}
});
it("waits for Refine notes before starting a revision turn", async () => {
let submitNotes: (notes: string) => void;
const flow = setup(["Refine"], [], () => new Promise((resolve) => {
submitNotes = resolve;
}));
try {
await flow.commands.get("goals").handler("objective", flow.ctx);
const planPath = join(flow.cwd, ".pi/plan/session-a-v1.md");
writeFileSync(planPath, "# Plan\n\n## Goals\n\n1. [ ] goal: make this specific\n");
const review = flow.hooks.get("agent_settled")({}, flow.ctx);
await new Promise((resolve) => setImmediate(resolve));
expect(flow.events).toEqual(["display", "select", "editor"]);
expect(flow.messages.filter((message) => !message.display)).toHaveLength(1);
submitNotes!("Name the output artifact.");
await review;
expect(flow.messages.at(-1)?.content).toContain("Revise the plan at");
} finally {
rmSync(flow.cwd, { recursive: true, force: true });
}
});
it("starts work only when the human chooses Ready", async () => {
const flow = setup(["Ready"]);
try {
@@ -140,6 +167,7 @@ describe("/goals draft flow", () => {
const writePlan = await flow.hooks.get("tool_call")({ toolName: "write", input: { path: planPath } }, flow.ctx);
const writeCode = await flow.hooks.get("tool_call")({ toolName: "write", input: { path: "README.md" } }, flow.ctx);
const readShell = await flow.hooks.get("tool_call")({ toolName: "bash", input: { command: "pwd && ls && git log" } }, flow.ctx);
const changeDirectoryThenRead = await flow.hooks.get("tool_call")({ toolName: "bash", input: { command: "cd . && ls -la" } }, flow.ctx);
const pipeShell = await flow.hooks.get("tool_call")({ toolName: "bash", input: { command: "ls | head" } }, flow.ctx);
const pythonWrite = await flow.hooks.get("tool_call")({ toolName: "bash", input: { command: "python -c \"open('README.md', 'w')\"" } }, flow.ctx);
const signoff = await flow.tools.get("CompleteGoal").execute("", { goal: "work" }, undefined, undefined, flow.ctx);
@@ -149,6 +177,7 @@ describe("/goals draft flow", () => {
expect(writePlan).toBeUndefined();
expect(writeCode?.block).toBe(true);
expect(readShell).toBeUndefined();
expect(changeDirectoryThenRead).toBeUndefined();
expect(pipeShell?.block).toBe(true);
expect(pythonWrite?.block).toBe(true);
expect(signoff.isError).toBe(true);
+16
View File
@@ -0,0 +1,16 @@
import { describe, expect, it } from "vitest";
import { planDrafting, planningState } from "../src/prompts.js";
describe("planning prompt", () => {
it("requires fact finding or a focused question before a goal", () => {
expect(planDrafting).toContain("Use read-only tools to resolve discoverable facts.");
expect(planDrafting).toContain("ask the\nhuman one focused question");
expect(planDrafting).toContain("placeholder goal such as \"work out the thing\"");
expect(planDrafting).toContain("object and an observable result");
});
it("restores the same rule after compaction", () => {
expect(planningState(".pi/plan/test.md")).toContain("Use read-only tools to find facts yourself");
expect(planningState(".pi/plan/test.md")).toContain("one focused question");
});
});