pi-agent - Test Suite Usage Guide

Let me read through all the test files to understand their content and purpose.
Now let me read the remaining harness test files and utility files.
Let me read the remaining test files.
Now let me read the utility files and scratch test.
Now I have a thorough understanding of the entire test suite. Here is the usage guide:

pi-agent Test Suite Usage Guide

Architecture Overview

The test suite is organized in three concentric layers, from outermost (public API) to innermost (infrastructure):

test/
├── agent.test.ts              # Layer 1: Agent public API
├── agent-loop.test.ts         # Layer 1: Agent loop internals
├── e2e.test.ts                # Layer 1: End-to-end integration
├── harness/                   # Layer 2: Harness subsystem
│   ├── agent-harness.test.ts          # AgentHarness orchestrator
│   ├── agent-harness-stream.test.ts   # Stream/option configuration
│   ├── compaction.test.ts             # Context compaction logic
│   ├── session.test.ts                # Session abstraction (multi-backend)
│   ├── session-backends.test.ts       # Session store backends
│   ├── repo.test.ts                   # Repository concurrency & lifecycle
│   ├── tools.test.ts                  # Built-in tools (read/write/edit/bash)
│   ├── nodejs-env.test.ts             # Node.js execution environment
│   ├── truncate.test.ts               # Text truncation utilities
│   ├── prompt-templates.test.ts       # Prompt template loading
│   ├── skills.test.ts                 # Skill file loading
│   ├── system-prompt.test.ts          # System prompt formatting
│   ├── resource-formatting.test.ts    # Resource formatting helpers
│   ├── sqlite-migrations.test.ts      # SQLite schema migrations
│   ├── sqlite-branch-cache.test.ts    # SQLite branch cache layer
│   └── sqlite-node.test.ts            # SQLite store + FTS search
├── utils/                     # Layer 3: Shared test utilities
│   ├── calculate.ts           # Calculator tool helper
│   └── get-current-time.ts    # Time tool helper
├── scratch/
│   └── simple.ts              # Standalone scratch script
└── harness/
    ├── session-test-utils.ts  # Session test factories
    └── tool-context.types.ts  # Compile-time type assertion

Layer 1: Agent Public API (Top-Level)

These three files test the Agent class and its core loop from the outside in.

1.1 `agent.test.ts` — Agent class unit tests

What it tests: The Agent class — construction, state management, event subscription, lifecycle, message queuing, and abort behavior.

Key scenarios:

Construction & state: Default state (empty system prompt, off thinking, no tools); custom initial state.
Event subscription: Subscribe/unsubscribe; async subscriber barrier (prompt waits for async listeners).
Lifecycle events: Full event sequence for thrown failures (agent_start → turn_start → message_start/end → turn_end → agent_end).
Abort: Abort signal propagation to subscribers; waitForIdle() awaiting async subscribers.
Tool update lifecycle: Late tool updates after settlement are silently ignored (no unhandled rejections); parallel tool with one settled + one running.
State mutators: systemPrompt, model, thinkingLevel, tools (defensive copy), messages (defensive copy).
Message queuing: steer() and followUp() queue messages; prompt() rejects while streaming; continue() rejects while streaming.
Continue semantics: Processes queued follow-ups; maintains one-at-a-time steering from assistant tail.
Legacy compatibility: prepareNextTurn signal callback; sessionId forwarding to stream function.

When to read: When you change the Agent class API surface, event emission, state management, or message queuing logic.

1.2 `agent-loop.test.ts` — Agent loop internals

What it tests: The agentLoop() and agentLoopContinue() functions — the core turn-taking engine that drives LLM calls, tool execution, and context management.

Key scenarios:

Event emission with AgentMessage types: Verifies event sequence and message ordering.
Custom message types via convertToLlm: Filtering/transformation of custom roles.
transformContext: Applied before convertToLlm for context pruning.
Tool calls & results: Execution, usage tracking, afterToolCall hook with usage patching.
Length-truncated assistant messages: Tool calls from stopReason: "length" are NOT executed; loop continues.
prepareArguments: Tool argument preparation before validation.
Parallel tool execution: Completion-order events but source-order persistence.
Steering message injection: Queued messages injected after all tool calls complete.
Execution mode: executionMode: "sequential" on a tool forces sequential even when config is parallel; mixed tools force sequential.
prepareNextTurn: Snapshot context modification before continuing.
shouldStopAfterTurn: Stops after current turn; skips follow-up polling.
terminate flag: Per-tool and per-batch termination; afterToolCall can mark batch as terminating.
agentLoopContinue: Throws on empty context; continues without user message events; supports custom message types.

When to read: When you modify the agent loop, tool execution pipeline, context transformation, or turn-management hooks.

1.3 `e2e.test.ts` — End-to-end integration

What it tests: Full Agent class with a real (faux) provider — no mocks at the stream level.

Key scenarios:

Basic prompt: User message → assistant response with content verification.
Tool execution: Calculator tool call → tool result → final answer; pendingToolCalls tracking.
Abort during streaming: Mid-stream abort → stopReason: "aborted".
Lifecycle updates: Event ordering during streaming (agent_start → message_update → message_end → agent_end).
Multi-turn conversation: Context preservation across turns.
Thinking content: Thinking blocks preserved in messages.
Agent.continue(): Validation (no messages, last message is assistant); continue from user message; continue from tool result.

When to read: When you need to verify the full Agent pipeline works end-to-end, or when adding new provider-level behavior.

Layer 2: Harness Subsystem

The harness wraps the Agent with session persistence, hooks, tools, and resource management.

2.1 `agent-harness.test.ts` — AgentHarness orchestrator

What it tests: The AgentHarness class — the primary orchestration layer that manages sessions, hooks, queue modes, shutdown, compaction, retries, and tool/resource management.

Key scenarios:

Construction: Queue modes (steering/follow-up), model/thinking level getters.
Shutdown lifecycle: requestShutdown() → aborts active work → waitForShutdown() settles; idempotent shutdown; rejects post-shutdown operations; hook/subscriber-initiated shutdown without deadlock.
Shutdown during hooks: before_agent_start hook blocks until released; shutdown during hook aborts prompt but not the harness.
Shutdown with compaction/tree navigation: Abort active compaction; abort tree navigation without moving leaf.
Concurrent mutations: appendMessage during active run; idle session mutations before shutdown.
Steering/follow-up drain: One-at-a-time mode with queue update events.
Hooks: before_agent_start messages injected and persisted; context hook failures settle with error messages.
Save-point refresh: Model, thinking level, resources, system prompt, and active tools refresh at save points (between turns).
Listener write ordering: Pending listener writes after agent-emitted messages.
waitForIdle: Waits for external run settlement and awaited listeners.
Tool hooks: tool_call and tool_result hooks with usage patching and termination.
Tool context: Static and async tool context providers.
Compaction: Generated and hook-provided compaction usage; retry events.
Branch summary: Generated and hook-provided branch summary usage.
Tool/resource management: setTools, setActiveTools, setResources with update events; validation (unknown/duplicate tools).
App type preservation: Generic type parameters for skills, prompt templates, and tools.

When to read: When you change the AgentHarness class, hooks, queue management, shutdown logic, or tool/resource APIs.

2.2 `agent-harness-stream.test.ts` — Stream configuration

What it tests: Stream options handling — snapshotting, chaining, deletion semantics, and save-point updates.

Key scenarios:

Snapshot before hooks: Base options snapshotted; before_provider_request hook merges headers/metadata.
Chaining patches: Multiple hooks chain; undefined values delete keys.
Save-point updates: Updated stream options used for next turn without mutating active request.
Payload hooks: before_provider_payload hook chains.

When to read: When you modify stream option handling or provider request hooks.

2.3 `compaction.test.ts` — Context compaction logic

What it tests: The compaction (summarization) system — token estimation, cut-point selection, summary generation, and split-turn compaction.

Key scenarios:

Token calculation: calculateContextTokens, estimateTokens across all message roles (user, assistant with thinking/tool calls, custom, toolResult with images, bashExecution, branchSummary, compactionSummary).
Threshold check: shouldCompact with enabled/disabled settings.
Cut-point selection: findCutPoint based on token differences; edge cases (thinking/model-change entries, branch summaries, custom messages, tool results, existing compaction entries).
Session context building: buildSessionContext with compaction entries; retained tail; model/thinking level tracking.
Compaction preparation: Previous summary extraction; split-turn with file operation details; custom/branch summary entries; empty/already-compacted sessions.
Serialization: Tool result truncation in serialized conversations.
Summary generation: Reasoning model support; previous summaries in prompts; custom instructions; error/abort handling; maxTokens clamping.
Compaction execution: Split-turn usage combination; turn-prefix reasoning; error results without throwing; file detail preservation.

When to read: When you modify compaction/summarization logic, token estimation, or context window management.

2.4 `session.test.ts` — Session abstraction (multi-backend)

What it tests: The Session interface — message appending, context building, branching, compaction, labels, and persistence across two backends (in-memory and JSONL).

Key scenarios:

Message append & context: Order-preserving append; buildContext returns messages in order.
Entry reading: Forward pagination with afterEntrySeq and limit.
Model/thinking tracking: appendModelChange, appendThinkingLevelChange reflected in context.
Branching: moveTo + append creates new branch; getBranch returns correct path.
Compaction: appendCompaction with retained tail; summary reconstruction in context.
Branch summary: moveTo with summary creates branch summary entry.
Compaction/branch usage persistence.
Custom messages: appendCustomMessageEntry in context; appendCustomEntry with projectors; entryTransforms.
Labels & session name: appendLabel, getSessionName (normalized).
Persistence: Reload from store preserves all state.

When to read: When you change the Session interface, context building, or add new entry types.

2.5 `session-backends.test.ts` — Session store backends

What it tests: Session aggregate operations and JsonlSessionStore specifics.

Key scenarios:

Aggregate operations: Leaf navigation, labels, names, stats, branch traversal.
Concurrent appends: Serialized into one parent chain.
Usage statistics: Assistant and summary usage in stats.
Branch traversal stop: At retained-tail compaction.
JSONL store: Header/entry writing; reopen; metadata persistence; malformed file handling; entry uniqueness; scope uniqueness; non-object header metadata rejection.

When to read: When you modify session store implementations or the session aggregate.

2.6 `repo.test.ts` — Repository concurrency & lifecycle

What it tests: SessionRepository — CRUD operations, forking, concurrency control, and disposal.

Key scenarios:

In-memory store: Open/delete/fork by metadata; full-session fork delegation; scoped reads without reload; context from branch reader; disposal rejects operations; await using support.
JSONL store: Conflicting create/fork serialization; encoded session IDs; concurrent appends to different sessions; concurrency cap (default 4, configurable); capacity release after failure; same-session serialization; listing as barrier; disposal waits for accepted operations; parse-once with retained state; cwd-scoped storage; malformed file handling; missing leaf rejection; fork with metadata persistence.

When to read: When you modify repository logic, concurrency control, or add new store backends.

2.7 `tools.test.ts` — Built-in tools

What it tests: The four built-in tools: read, write, edit, bash.

Key scenarios:

Read: Offset/limit/continuation; large file truncation; trailing newline edge case; offset beyond file; image detection (PNG, BMP with conversion); injected image processor.
Write: File creation with parent directories; mutation queue lock during abort.
Edit: Disjoint edits with diff/patch output; overlap rejection; missing/duplicate target text; mutation queue lock during abort; symlink serialization; BOM/CRLF preservation.
Bash: Command execution with stdout/stderr; nonzero exit/timeout; truncated output on timeout; late output ignored; oversized line reporting; prepare hook (command prefix, cwd, env); update coalescing; full output persistence.

When to read: When you modify any built-in tool implementation.

2.8 `nodejs-env.test.ts` — Node.js execution environment

What it tests: NodeExecutionEnv — file system operations, shell execution, and cleanup.

Key scenarios:

File operations: Read/write/list/remove; home-relative and file URL expansion; file info (file/dir/symlink); symlinks in listings; text line limits; missing path errors; append with parent dirs; temp dirs/files; recursive/non-recursive create; force/non-force remove.
Abort support: Pre-aborted file operations return aborted results.
Shell execution: CWD and env overrides; inherit/replace env; stdin transport for WSL; detached descendant settlement; cleanup terminates processes; stdout/stderr streaming; missing CWD error; non-zero exit codes; timeout; callback errors; shell unavailable/spawn errors; aborted commands.
Large output capture: executeShellWithCapture with truncation.

When to read: When you modify NodeExecutionEnv or shell execution logic.

2.9 `truncate.test.ts` — Text truncation utilities

What it tests: truncateHead and truncateTail — byte-aware, line-aware text truncation.

Key scenarios:

UTF-8 byte counting; trailing newline edge cases; head/tail truncation on byte limits; first-line-exceeds-limit; last-line-partial; oversized single line; surrogate edge cases; deterministic fuzz testing against Buffer.subarray semantics.

When to read: When you modify truncation logic.

2.10 `prompt-templates.test.ts` — Prompt template loading

What it tests: Loading .md prompt templates from directories.

Key scenarios: Non-recursive loading from multiple dirs; YAML frontmatter parsing; first-line description fallback; source info preservation; diagnostic source info; explicit files and symlinks; argument substitution ($1, ${@:2}, $ARGUMENTS).

When to read: When you modify prompt template loading or formatting.

2.11 `skills.test.ts` — Skill file loading

What it tests: Loading SKILL.md files from skill directories.

Key scenarios: SKILL.md with YAML frontmatter; symlinked directories; source info preservation; diagnostic source info; direct markdown children only from root; disable-model-invocation flag.

When to read: When you modify skill loading logic.

2.12 `system-prompt.test.ts` — System prompt formatting

What it tests: formatSkillsForSystemPrompt — XML formatting of skills in system prompts.

Key scenarios: Visible skills in order; model-disabled skills skipped; XML escaping; empty skill list.

When to read: When you modify system prompt skill formatting.

2.13 `resource-formatting.test.ts` — Resource formatting helpers

What it tests: formatSkillInvocation and formatPromptTemplateInvocation — how skills and templates are formatted when invoked.

Key scenarios: Skill invocation with additional instructions; prompt template with positional arguments.

When to read: When you modify resource invocation formatting.

2.14 `sqlite-migrations.test.ts` — SQLite schema migrations

What it tests: SQLite database migrations, session CRUD, forking, branch materialization, and summary state.

Key scenarios: File-based migration application; legacy branch projection cleanup; session metadata persistence through CRUD/fork; fork rollback on failure; active leaf materialization; new branch materialization; reopen with branch materialization; entry pagination; database close on create/open failure; connection retention; shared connection across source/fork; missing leaf rejection; malformed entry handling; transaction rollback on append failure; session summary field materialization.

When to read: When you modify SQLite migrations or the SQLite session store.

2.15 `sqlite-branch-cache.test.ts` — SQLite branch cache

What it tests: The branch cache layer that stores pre-computed branch paths for fast traversal.

Key scenarios: Complete root paths after compaction; compacted branch window reading; nested compaction boundaries; missing cache repair from canonical links; interrupted repair rollback; fork with source cache repair; branch entry/tip deletion with session.

When to read: When you modify the SQLite branch cache layer.

2.16 `sqlite-node.test.ts` — SQLite store + FTS search

What it tests: SqliteSessionStore with FTS5 full-text search, and ScanningSessionSearch for JSONL.

Key scenarios: Scanning search for JSONL; SQLite FTS5 search; empty session without FTS init; FTS trigger failure rollback (append and delete); search-before-create initialization; custom search implementation swap.

When to read: When you modify SQLite search or add new search backends.

Layer 3: Support Files

3.1 `utils/calculate.ts` — Calculator tool helper

A shared test utility that provides a calculateTool (evaluates math expressions) and createCalculateToolWithUsage (with custom usage tracking). Used by e2e.test.ts and agent-harness.test.ts.

3.2 `utils/get-current-time.ts` — Time tool helper

A shared test utility providing getCurrentTimeTool (gets current date/time with optional timezone). Used by agent-harness.test.ts for multi-tool tests.

3.3 `session-test-utils.ts` — Session test factories

Shared factories: createInMemorySession(), createUserMessage(), createAssistantMessage(), createTempDir(). Used by most harness tests. Automatically cleans up temp directories in afterEach.

3.4 `tool-context.types.ts` — Compile-time type assertion

A type-only file (no runtime tests) that verifies AgentHarness enforces toolContext when tools require it. Fails at compile time if the constraint is broken.

3.5 `scratch/simple.ts` — Standalone scratch script

A standalone script (not a test) for manual experimentation with the Agent API.

Running Tests

# All tests
npx vitest run pi-agent/test/

# Single file
npx vitest run pi-agent/test/agent.test.ts

# Watch mode
npx vitest watch pi-agent/test/

# With coverage
npx vitest run pi-agent/test/ --coverage

这个测试使用的是 Vitest（不是 Vite，虽然 Vitest 是基于 Vite 构建的）。

测试文件第 3 行明确导入了 vitest：

import { describe, expect, it } from "vitest";

运行方式

在 pi-agent 目录下，有两种方式：

方式 1：通过 pnpm script

cd c:\Users\lukel\Documents\apps\harness\pi-agent
pnpm test

这会执行 [package.json](file:///c:/Users/lukel/Documents/apps/harness/pi-agent/package.json) 中定义的 vitest --run。

方式 2：直接运行 vitest 并指定文件

cd c:\Users\lukel\Documents\apps\harness\pi-agent
pnpm vitest --run test/agent.test.ts

--run 参数表示只运行一次测试，不进入 watch 模式。

方式 3：从 monorepo 根目录运行

cd c:\Users\lukel\Documents\apps\harness
pnpm --filter @earendil-works/pi-agent-core test

注意事项

项目使用 pnpm workspace，需要先在根目录执行过 pnpm install 安装依赖
测试配置在 [vitest.config.ts](file:///c:/Users/lukel/Documents/apps/harness/pi-agent/vitest.config.ts)，设置了 environment: "node" 和 30 秒超时
还有一个 test:harness 脚本用的是 [vitest.harness.config.ts](file:///c:/Users/lukel/Documents/apps/harness/pi-agent/vitest.harness.config.ts) 配置，但默认的 pnpm test 跑的是标准 vitest 配置

posted @ 2026-09-18 00:14 lukefoto 阅读(16) 评论(0) 收藏举报

刷新页面返回顶部

ziitoo

pi-agent - Test Suite Usage Guide

pi-agent Test Suite Usage Guide

Architecture Overview

Layer 1: Agent Public API (Top-Level)

1.1 `agent.test.ts` — Agent class unit tests

1.2 `agent-loop.test.ts` — Agent loop internals

1.3 `e2e.test.ts` — End-to-end integration

Layer 2: Harness Subsystem

2.1 `agent-harness.test.ts` — AgentHarness orchestrator

2.2 `agent-harness-stream.test.ts` — Stream configuration

2.3 `compaction.test.ts` — Context compaction logic

2.4 `session.test.ts` — Session abstraction (multi-backend)

2.5 `session-backends.test.ts` — Session store backends

2.6 `repo.test.ts` — Repository concurrency & lifecycle

2.7 `tools.test.ts` — Built-in tools

2.8 `nodejs-env.test.ts` — Node.js execution environment

2.9 `truncate.test.ts` — Text truncation utilities

2.10 `prompt-templates.test.ts` — Prompt template loading

2.11 `skills.test.ts` — Skill file loading

2.12 `system-prompt.test.ts` — System prompt formatting

2.13 `resource-formatting.test.ts` — Resource formatting helpers

2.14 `sqlite-migrations.test.ts` — SQLite schema migrations

2.15 `sqlite-branch-cache.test.ts` — SQLite branch cache

2.16 `sqlite-node.test.ts` — SQLite store + FTS search

Layer 3: Support Files

3.1 `utils/calculate.ts` — Calculator tool helper

3.2 `utils/get-current-time.ts` — Time tool helper

3.3 `session-test-utils.ts` — Session test factories

3.4 `tool-context.types.ts` — Compile-time type assertion

3.5 `scratch/simple.ts` — Standalone scratch script

Recommended Reading Order

Running Tests

运行方式

注意事项

公告

ziitoo

pi-agent - Test Suite Usage Guide

pi-agent Test Suite Usage Guide

Architecture Overview

Layer 1: Agent Public API (Top-Level)

1.1 agent.test.ts — Agent class unit tests

1.2 agent-loop.test.ts — Agent loop internals

1.3 e2e.test.ts — End-to-end integration

Layer 2: Harness Subsystem

2.1 agent-harness.test.ts — AgentHarness orchestrator

2.2 agent-harness-stream.test.ts — Stream configuration

2.3 compaction.test.ts — Context compaction logic

2.4 session.test.ts — Session abstraction (multi-backend)

2.5 session-backends.test.ts — Session store backends

2.6 repo.test.ts — Repository concurrency & lifecycle

2.7 tools.test.ts — Built-in tools

2.8 nodejs-env.test.ts — Node.js execution environment

2.9 truncate.test.ts — Text truncation utilities

2.10 prompt-templates.test.ts — Prompt template loading

2.11 skills.test.ts — Skill file loading

2.12 system-prompt.test.ts — System prompt formatting

2.13 resource-formatting.test.ts — Resource formatting helpers

2.14 sqlite-migrations.test.ts — SQLite schema migrations

2.15 sqlite-branch-cache.test.ts — SQLite branch cache

2.16 sqlite-node.test.ts — SQLite store + FTS search

Layer 3: Support Files

3.1 utils/calculate.ts — Calculator tool helper

3.2 utils/get-current-time.ts — Time tool helper

3.3 session-test-utils.ts — Session test factories

3.4 tool-context.types.ts — Compile-time type assertion

3.5 scratch/simple.ts — Standalone scratch script

Recommended Reading Order

Running Tests

运行方式

注意事项

公告

1.1 `agent.test.ts` — Agent class unit tests

1.2 `agent-loop.test.ts` — Agent loop internals

1.3 `e2e.test.ts` — End-to-end integration

2.1 `agent-harness.test.ts` — AgentHarness orchestrator

2.2 `agent-harness-stream.test.ts` — Stream configuration

2.3 `compaction.test.ts` — Context compaction logic

2.4 `session.test.ts` — Session abstraction (multi-backend)

2.5 `session-backends.test.ts` — Session store backends

2.6 `repo.test.ts` — Repository concurrency & lifecycle

2.7 `tools.test.ts` — Built-in tools

2.8 `nodejs-env.test.ts` — Node.js execution environment

2.9 `truncate.test.ts` — Text truncation utilities

2.10 `prompt-templates.test.ts` — Prompt template loading

2.11 `skills.test.ts` — Skill file loading

2.12 `system-prompt.test.ts` — System prompt formatting

2.13 `resource-formatting.test.ts` — Resource formatting helpers

2.14 `sqlite-migrations.test.ts` — SQLite schema migrations

2.15 `sqlite-branch-cache.test.ts` — SQLite branch cache

2.16 `sqlite-node.test.ts` — SQLite store + FTS search

3.1 `utils/calculate.ts` — Calculator tool helper

3.2 `utils/get-current-time.ts` — Time tool helper

3.3 `session-test-utils.ts` — Session test factories

3.4 `tool-context.types.ts` — Compile-time type assertion

3.5 `scratch/simple.ts` — Standalone scratch script