Skip to content

Make the agent harness pluggable with AI SDK Harnesses #181

Open
Open
Make the agent harness pluggable with AI SDK Harnesses#181
Assignees
lavaman131

Description

@lavaman131
lavaman131
opened on Sep 24, 2026

Pluggable agent harness on AI SDK Harnesses: Technical Design Document / RFC

Document Metadata Details
Author(s) lavaman131
Status Draft (WIP)
Team / Owner Chopin server (apps/server/src/agent)
Created / Last Updated 2026-09-24
Compatibility posture Internal server APIs may break freely; existing deployments keep working unchanged
Related #157 (local device-flow sign-in), #56, #80

1. Executive Summary

The Planner, background workers, and public research workers run directly on
@github/copilot-sdk, and SDK types reach into chat, jobs, tools, and
permission code. Chopin is open source and hosted by whoever deploys it, so it
shouldn't depend on one vendor's harness. Copilot SDK also can't provide some
capabilities we want in the sidebar chat, such as a classifier model and image
generation.

This RFC adopts Vercel's AI SDK Harnesses
(@ai-sdk/harness) as Chopin's harness layer instead of writing our own.
HarnessV1 is the adapter contract, HarnessAgent runs sessions, and each
harness owns its own agent loop. Adapters such as Pi, Claude Code, and Codex
come from their catalog.

The split is simple: the harness brings the agent loop and the model; Chopin
brings the tools. Document tools, repository tools, and GitHub pull-request
tools are Chopin's and reach every harness as host-executed AI SDK tools bound
to the channel's repository. Every harness built-in stays inactive. Copilot stays
the default through a small Chopin-owned HarnessV1 adapter over the in-process
Copilot SDK.

2. Context and Motivation

2.1 Current State

apps/server/src
├── agent/client.ts        CopilotClient, hardened SessionConfig, open*/discard/abort/settle/shutdown, tool audits
├── agent/runtime.ts       Runtime generations over CopilotClient/CopilotSession
├── agent/permissions.ts   gate / terminalGate / publicResearchGate over SDK PermissionRequest
├── agent/planner.ts       CustomAgentConfig + TOOLS filter ("mcp:*", "custom:*")
├── agent/tools.ts         Planner document tools typed as SDK Tool
├── agent/repository.ts    read_repository_file, list_repository_tree, search_repository, repository_history
├── chat/service.ts        agent.session.on/send, translate(SessionEvent)
├── jobs/document-summary.ts   CopilotSummaryEngine, submit_job_result terminal tool
└── jobs/research-workspace.ts agent.session.on/send, terminal result tool
  • Leaking doors (today):
    • Agent.Agent exposes the raw CopilotSession, so callers reach into
      session.on / session.send and SDK event names.
    • Chopin's tools are typed as the Copilot SDK's Tool, so the tools that
      define what Chopin is can't be handed to another runtime.
    • The Planner's tool audit only logs, while worker opens fail closed.
    • The runtime connects to GitHub MCP with the owner's user token, which can
      read every installation repository the owner can access. Repository scope
      is enforced afterward by inspecting MCP call arguments (hasForeignScope).

2.2 The Problem

  • Product: self-hosted deployments can't bring their own runtime, and
    classifier and image-generation capabilities are unreachable.
  • Positioning: open source, self-hosted Chopin is coupled to one vendor.
  • Technical debt: Chopin's tools and security boundary are written in one
    SDK's vocabulary.

3. Goals and Non-Goals

3.1 Functional Goals

  • @ai-sdk/harness (HarnessV1, HarnessAgent) is the only harness
    abstraction. Chopin defines no parallel port types.
  • @github/copilot-sdk is imported only by the Chopin Copilot SDK adapter,
    enforced by a lint rule.
  • HARNESS selects an adapter factory from Chopin's harness map. The default
    is the Copilot SDK adapter.
  • Chopin's document, repository, and GitHub tools are AI SDK tool()s,
    executed in the Chopin process and bound to the channel's repository
    through toolsContext.
  • Every harness built-in is inactive for every Chopin agent.
  • GitHub MCP is reached only through createMCPClient in the Chopin
    process. No harness adapter receives a GitHub credential or an
    mcpServers entry.
  • Workers use HarnessAgent structured output.
  • @ai-sdk/harness-pi passes the contract suite as the second adapter.
    The author's runtime follows as its own HarnessV1 package and carries a
    harness-provided classifier model end to end.

3.2 Non-Goals (Out of Scope)

  • No Chopin-defined harness port, capability object, event union, or
    permission request type. We use AI SDK's.
  • No Chopin-owned agent loop. Harnesses own their loops; Chopin adds no
    ToolLoopAgent or provider-model path.
  • No harness built-ins (read, bash, write, edit, web fetch, and so
    on). The Planner still has no checkout, shell, or host filesystem.
  • No resumable sessions: no detach, stop, resumeFrom, or
    continueFrom. Sessions are destroyed after each turn.
  • No human approval pauses. Permissions are configuration only.
  • No harness that hasn't passed the contract suite, and no runtime loading
    of adapters by package name.
  • No GitHub App private key or installation tokens.
  • No change to the MCP (/mcp) coding-agent boundary.
  • No new sign-in flow. Local sign-in is Support device-flow sign-in for local instances with persistent credentials #157's scope.
  • Not in scope: restricting image hosts in documents. Any https: image
    renders today (widgets/image.tsx), which is a pre-existing exfiltration
    path tracked separately.
  • Not in scope: storing or serving harness-generated images in documents.

4. Proposed Solution (High-Level Design)

4.1 System Architecture Diagram

flowchart TB
    Chat[chat/service] --> Agents
    Summary[jobs/document-summary] --> Agents
    Research[jobs/research-workspace] --> Agents
    Agents["Chopin agents<br/>HarnessAgent configs<br/>(planner, summary, research)"]
    Agents --> HA["@ai-sdk/harness<br/>HarnessAgent · agent loop owned by harness"]
    HA --> Map{"harness map<br/>(HARNESS)"}
    Map --> CopilotSdk["Chopin Copilot SDK adapter<br/>host process"]
    Map --> Pi["@ai-sdk/harness-pi<br/>host process"]
    Map -.-> Other["other tested adapters"]
    Tools["Chopin tools (host-executed)<br/>document · repository · GitHub MCP<br/>bound via toolsContext"] --> HA
    MCP["@ai-sdk/mcp createMCPClient<br/>GitHub MCP, owner token"] --> Tools
    REST["GitHub REST<br/>owner token"] --> Tools
Loading

The airlock is openPlannerSession: the only place a session receives tools,
and every tool it receives is bound to one repository. GitHub credentials stay
in the Chopin process.

4.2 Architectural Pattern

AI SDK's harness pattern, used as documented. An adapter factory
(createPi(), createCopilotSdk()) produces a HarnessV1, and a
HarnessAgent built from it is constructed once at module scope. Harnesses own
their agent loops and models. Chopin supplies instructions, host tools,
activeTools, and output, and consumes the resulting stream. Selecting a
harness is a map from the HARNESS name to an adapter factory.

Tools that express Chopin's domain (documents, repositories, pull requests) are
Chopin's, not the harness's. Capabilities that belong to a runtime (a
classifier model, image generation) come from the harness.

4.3 Key Components

Component Responsibility Source
HarnessAgent, HarnessV1 Sessions, agent loop, tool filtering, stream parts, structured output @ai-sdk/harness
MCP client GitHub MCP tools as AI SDK tools @ai-sdk/mcp
Harness selection HARNESS name → adapter factory; startup checks Chopin
Chopin agents Planner, summary, and research HarnessAgent configurations Chopin
Chopin tools Document, repository, and GitHub tools as AI SDK tool()s Chopin
Copilot SDK adapter Host-process HarnessV1 over @github/copilot-sdk Chopin
Conversation projection AI SDK stream parts → Chat.* protocol messages Chopin
Contract suite Checks each adapter in the map against Chopin's boundary Chopin
Path Action Owns
apps/server/src/harness/harnesses.ts add harnesses name → factory map; harnessFor(config); startup checks
apps/server/src/harness/agents.ts add plannerAgent, summaryAgent, researchAgent definitions
apps/server/src/harness/session.ts add openPlannerSession; empty sandbox; toolsContext
apps/server/src/harness/github-tools.ts add githubTools (MCP client, schemas allowlist, bindRepository)
apps/server/src/harness/contract.ts add Contract suite run against every adapter in the map
apps/server/src/harness/copilot-sdk/adapter.ts add (from agent/client.ts) createCopilotSdk() implementing HarnessV1
apps/server/src/harness/copilot-sdk/runtime.ts move (from agent/runtime.ts) Copilot runtime generations
apps/server/src/agent/tools.ts change Document tools as AI SDK tool() with contextSchema
apps/server/src/agent/repository.ts change Same four tools as AI SDK tool(); repository from toolsContext
apps/server/src/agent/permissions.ts, agent/planner.ts delete / shrink Replaced by activeTools, permissionMode, and agent instructions
apps/server/src/agent/client.ts, agent/runtime.ts delete Moved into the Copilot SDK adapter
apps/server/src/chat/service.ts change agent.stream(); translate(room, part) over AI SDK stream parts
apps/server/src/jobs/document-summary.ts, research-workspace.ts change HarnessAgent with output schemas
apps/server/src/config.ts change HARNESS, HARNESS_AUTH
.oxlintrc.json change Restrict @github/copilot-sdk to the adapter directory
e2e/ change Fake HarnessV1 and fake GitHub MCP server for harness-path coverage
docs/hosted-agent.md, docs/self-hosting.md, docs/architecture.md change Harness selection, adapter trust, host-side GitHub tools

4.4 The Door Set at a Glance

Chopin's own doors: harnessFor, githubTools ⚠, openPlannerSession ⚠,
translate.

Doors used from AI SDK: createMCPClient, mcpClient.tools,
HarnessAgent.createSession, HarnessAgent.stream, HarnessAgent.generate,
session.destroy.

Read together: a deployment picks one tested harness. A Planner session is
opened only with Chopin's tools, each bound to the channel's repository. The
harness runs the loop, and Chopin projects what happened into the Conversation.
Sessions never outlive a turn, and GitHub credentials never leave the Chopin
process.

5. Detailed Design

5.1 The Doors (Entrypoint Contracts)

import type { HarnessV1 } from "@ai-sdk/harness";
import { HarnessAgent } from "@ai-sdk/harness/agent";
import { createMCPClient } from "@ai-sdk/mcp";
import { Output, tool } from "ai";

// — Harness selection: a name → factory map, as AI SDK adapters are meant to be used. —

const harnesses = {
	"copilot-sdk": createCopilotSdk,
	pi: createPi,
	// adding an entry requires passing the contract suite and a security review
} satisfies Record<string, (settings: HarnessSettings) => HarnessV1>;

type HarnessConfig = {
	harness: keyof typeof harnesses;
	model?: string; // harness-specific model id, passed to HarnessAgent
	settings: HarnessSettings; // forwarded verbatim to the factory, including `auth`
};

function harnessFor(config: HarnessConfig, deployment: DeploymentMode): HarnessV1;
// Guarantee: returns the configured adapter from the harness map.
// Refuses at startup: unknown harness; an auth mode that falls back to a subscription
// logged in on the host (for example "auto") unless the server is local-only.

// — Chopin tools: host-executed, bound to the channel's repository through toolsContext. —

let repositoryTools = {
	read_repository_file, // today's tools, ported from SDK Tool to AI SDK tool();
	list_repository_tree, // no input field names a repository; the handler reads it
	search_repository, // from context.repository.
	repository_history,
};

const GITHUB_TOOL_SCHEMAS = {
	// `schemas` loads only these tools. Schemas omit owner and repo; descriptions are ours.
	// Today's effective set: the read-only pull_requests toolset minus search_pull_requests,
	// which the current gate already denies.
	pull_request_read: {
		inputSchema: z.object({
			method: z.enum([
				"get",
				"get_diff",
				"get_status",
				"get_files",
				"get_commits",
				"get_review_comments",
				"get_reviews",
				"get_comments",
				"get_check_runs",
			]),
			pullNumber: z.number(),
			page: z.number().optional(),
			perPage: z.number().optional(),
			after: z.string().optional(),
		}),
	},
	list_pull_requests: {
		inputSchema: z.object({
			state: z.string().optional(),
			base: z.string().optional(),
			head: z.string().optional(),
			sort: z.string().optional(),
			direction: z.string().optional(),
			page: z.number().optional(),
			perPage: z.number().optional(),
		}),
	},
};

function githubTools(owner: ActiveOwnerBinding): Promise<Result<GitHubTools, GitHubToolsError>>;
// Guarantee: returns host-executed AI SDK tools that can only address the repository
// supplied at call time through toolsContext.
// Steps: createMCPClient({ transport: { type: "http", url, headers: owner token, read-only,
// toolsets } }) → mcpClient.tools({ schemas: GITHUB_TOOL_SCHEMAS }) → bindRepository.
// GitHubToolsError = Unavailable | MissingTool(name)
// Refusal: no schema has an owner or repo field, and bindRepository overwrites both from
// context, so a model cannot name another repository. Search tools are not in the allowlist.

function bindRepository(tools: ToolSet, descriptions: Record<string, string>): ToolSet;
// For each tool: tool({ description: ours, inputSchema, contextSchema: RepositoryContext,
//   execute: (input, { context, ...rest }) =>
//     mcpTool.execute({ ...input, owner: context.owner, repo: context.repo }, rest) })

// — Agents: HarnessAgent configurations constructed once at module scope. —

let plannerAgent = new HarnessAgent({
	harness,
	model: config.model,
	tools: { ...documentTools, ...repositoryTools, ...githubTools },
	activeTools: [...documentToolNames, ...repositoryToolNames, ...githubToolNames], // no built-ins
	permissionMode: "allow-reads",
	instructions: plannerInstructions,
	prepareCall: ({ options, ...call }) => ({ ...call, instructions: withRepository(options) }),
});

let summaryAgent = new HarnessAgent({
	harness,
	activeTools: [],
	output: Output.object({ schema: DescriptionSchema }),
	instructions: summaryInstructions,
});

let researchAgent = new HarnessAgent({
	harness,
	tools: { web_search: publicWebSearch }, // host tool over GitHub MCP web_search
	activeTools: ["web_search"],
	output: Output.object({ schema: ResearchReportSchema }),
	instructions: researchInstructions,
});

// — Sessions. —

function openPlannerSession(
	owner: ActiveOwnerBinding,
	channel: ChannelRef,
): Promise<Result<PlannerSession, OpenError>>;
// Guarantee: returns a session whose only tools are Chopin's, bound to the channel's
// repository, with every harness built-in inactive.
// Steps: authorize owner → githubTools(owner) → empty just-bash sandbox →
// plannerAgent.createSession({ sandboxSession, toolsContext: { room, repository, owner } }).
// OpenError = GitHubToolsError | HarnessCapabilityUnsupported | Timeout | ShuttingDown

// — Projection. —

function translate(room: Room, part: HarnessStreamPart): void;
// Guarantee: projects one stream part into Chat.* protocol state.
// A tool-call for a tool outside activeTools aborts the turn and logs a boundary
// failure. This detects a broken adapter after the fact; it does not prevent the call.

Per-door audit:

Door Joint One sentence Every exit Refusals Trust transition Chokepoint
harnessFor ✅ choose the deployment's harness ✅ "returns the configured adapter" unknown harness or host-login fallback on a public server → startup failure untested adapters are not in the map's type n/a ✅ only place an adapter is constructed
githubTools ⚠ ✅ give the agent GitHub reads ✅ "returns GitHub tools bound to one repository" server down → Unavailable; tool missing → MissingTool no repository field in any schema; search tools not loaded ✅ owner token → repository-bound tools ✅ only path from a harness to GitHub MCP
openPlannerSession ⚠ ✅ start a Planner session ✅ "opens a session with only Chopin's bound tools" any step fails → sandbox destroyed, named error no built-ins; no unbound tools ✅ the only place a model gains tools ✅ only door to a Planner session
translate ✅ show the turn ✅ "projects one part into the Conversation" inactive tool → abort turn, boundary failure — n/a n/a

5.2 How Chopin Uses AI SDK

Chopin need AI SDK mechanism
Swap runtimes HarnessV1 adapter factories; HARNESS picks one from the map
Agent loop and model The harness, through HarnessAgent
Planner may only use Chopin tools activeTools lists only host tools; permissionMode: "allow-reads" as a second layer
Document and repository tools Host-executed AI SDK tool()s
GitHub pull requests and checks createMCPClient + mcpClient.tools({ schemas }), wrapped by bindRepository
Room, repository, owner context toolsContext with per-tool contextSchema; host-only, never serialized
Per-turn instructions prepareCall; settings fixed for the whole turn
Worker results output: Output.object({ schema })
Harness capabilities Built-ins, models, and stream parts the harness offers
Model credentials Adapter auth / credentials settings
Cancellation abortSignal on stream(), wired to the active owner's signal
Session end session.destroy() after every turn

Chopin configuration choices on top of AI SDK defaults:

  • activeTools lists only Chopin's host tools, so no built-in is active.
    permissionMode is allow-reads rather than the default allow-all as a
    second layer; any approval request the stream still produces is denied
    automatically, so no turn waits for a human.
  • Adapters receive no mcpServers. External MCP servers are consumed on the
    host with createMCPClient, so credentials and scope stay under Chopin's
    control on every adapter.
  • HarnessAgent requires a sandbox. Chopin passes an empty just-bash session
    that no active tool can reach.
  • detach, stop, resumeFrom, continueFrom, and suspendTurn are unused.

5.3 The Copilot SDK Adapter

The official @ai-sdk/harness-github-copilot adapter runs Copilot CLI inside a
network sandbox through ACP. Its documented limitations rule it out for the
Planner:

  • it can't filter built-ins ("filtering GitHub Copilot built-ins throws an
    unsupported-capability error");
  • it doesn't support built-in approval requests;
  • it needs a network sandbox with an exposed port.

Chopin therefore ships createCopilotSdk(), a host-process HarnessV1 that:

  • wraps @github/copilot-sdk in the Chopin process, moving today's
    client.ts hardening and runtime.ts generations inside it, so the Copilot
    CLI still runs as its own process over stdio;
  • declares no built-ins and registers only the host tools HarnessAgent
    passes, keeping today's live tool check against Copilot session metadata;
  • disables Copilot's built-in GitHub MCP, because GitHub tools arrive as host
    tools;
  • implements output internally with a result tool, since the Copilot SDK path
    has no structured output today, so Chopin's workers have one code path;
  • takes per-session Copilot credentials through a credentials resolver keyed
    by session ID and resolved from the active owner. Device-flow user tokens from
    Support device-flow sign-in for local instances with persistent credentials #157 work the same way as web-flow tokens here.

If AI SDK's Copilot adapter later supports built-in filtering, Chopin can
switch to it and delete this adapter.

5.4 GitHub Tools

  • Credential: the owner's App user token (web flow, or Support device-flow sign-in for local instances with persistent credentials #157's device flow)
    is used only by the host MCP client and the repository tools. No adapter or
    model sees it.
  • Scope: the schemas allowlist decides which GitHub MCP tools exist, and
    bindRepository decides which repository they address. The repository tools
    already take the repository from their options rather than their input. The
    read-only header and toolset selection stay as a second layer. The
    argument-inspecting gate (hasForeignScope) is deleted.
  • Descriptions: bindRepository supplies Chopin's own descriptions and
    schemas, so text from GitHub's server never reaches the model as tool
    metadata. Tool results still come from GitHub, as today.
  • Connections: the MCP client is cached per owner token and closed when the
    token rotates or expires, instead of reconnecting every turn.
  • Revalidation: the active owner's signal is the turn's abortSignal.
    Losing repository permission ends the turn.

5.5 Deployment Modes

  • Hosted (default): HARNESS_AUTH must name an explicit mode or supply
    credentials. A mode that falls back to a subscription logged in on the host is
    refused, so a server's own subscription is never lent to users.
  • Shared model keys: when HARNESS_AUTH supplies an operator key, every
    admitted writer's turns bill that key. Chopin adds no quotas in this revision;
    the self-hosting guide documents this and points operators to their
    provider's spend limits. Existing per-job bounds carry over as adapter
    settings (for example a maximum turn count, or the Copilot adapter's
    maxAiCredits).
  • Local: the harness uses its own signed-in state (for example
    auth: "auto"), so Chopin stores no model credentials. Host-login fallbacks
    are allowed only when the server is bound to a loopback interface. When Support device-flow sign-in for local instances with persistent credentials #157
    lands, this check reads its local-mode setting instead.

GitHub access is identical in both modes: the signed-in user's App token,
consumed on the host.

5.6 Data Model / Schema

No storage change. GitHub and model credentials are process-local.

5.7 Algorithms and State Management

  • Fixed per turn: prepareCall settles model, instructions, and tools when
    a turn starts.
  • No replay: an interrupted turn is never resent. The session and sandbox
    are destroyed.
  • Persistence before publication: host tools keep today's semantics.
  • In-process harnesses: host-process adapters such as Pi run their loop in
    the Chopin process. Their work is mostly model I/O, and Chopin's tools are
    I/O-bound API calls, so no large CPU work runs in the room process. Moving
    harness sessions into a worker is a later hardening if an adapter proves
    CPU-heavy.

6. Alternatives Considered

Option Pros Cons Decision
A: Keep Copilot SDK only No work Vendor coupling; no classifier or image generation Rejected
B: Chopin-owned port modeled on AI SDK Insulated from upstream changes Reimplements an existing, maintained layer; every adapter is ours to write Rejected
C: ACP as the primary integration One protocol for many agents Clients can't list or restrict the agent's own tools; AI SDK's ACP adapters inherit this Rejected for the Planner
D: Adopt @ai-sdk/harness; Chopin tools as host tools Maintained catalog; harness owns the loop; Chopin owns tools Experimental, fast-moving upstream API Selected
E: Official AI SDK Copilot adapter No Chopin adapter code Can't filter built-ins; needs a network sandbox Rejected for now
F: Repository snapshot with harness read/grep/glob Harness-native tools; regex search Copies the repository into the server; large repositories become unusable Rejected
G: GitHub MCP in each adapter's mcpServers Standard adapter setting The harness holds the token; scope depends on each adapter Rejected
H: Repository-scoped installation tokens GitHub enforces one-repository scope Needs the App private key, which conflicts with #157's local mode Rejected; not needed
I: A gh command in the sandbox Familiar CLI for models Requires active bash; command allowlists are weaker than typed schemas Rejected

7. Cross-Cutting Concerns

7.1 Security and Privacy

  • Chopin owns every tool: the harness sees only Chopin's host tools. None
    of their schemas can name a repository.
  • GitHub credentials stay in Chopin: the owner's token is used only by
    Chopin's tools and the host MCP client.
  • Adapter trust: builtinTools and supportsBuiltinToolFiltering are
    declarations the adapter makes about itself. The contract suite checks those
    declarations and the tools a session is given; it can't prove what the
    underlying runtime does internally. Adding an adapter to the map is a trust
    decision reviewed as a security change. The Copilot SDK adapter additionally
    checks the live session's tool metadata.
  • No lent subscriptions: host-login auth fallbacks are refused on servers
    that aren't loopback-only.
  • Credentials stay host-side: toolsContext and credentials are never
    serialized or shown to the model, and never enter PostgreSQL, logs, or browser
    storage.
  • Unchanged invariant: the Planner still has no checkout, shell, or host
    filesystem.

7.2 Backwards Compatibility

  • Default HARNESS is copilot-sdk. AGENT and MODEL keep their meaning.
    No new secrets or GitHub App permissions are required.
  • Chopin's tools keep their names and behavior; only their type changes.
  • Internal modules (agent/client.ts, agent/runtime.ts, agent/permissions.ts)
    are deleted without shims.

7.3 Upstream Churn

@ai-sdk/harness shipped 120 stable releases between 1.0.0 (2026-06-25) and
1.0.124 (2026-09-24), and its docs say to expect breaking changes. Chopin pins
exact versions of @ai-sdk/harness, @ai-sdk/mcp, and each adapter, upgrades
deliberately, and runs the contract suite on every bump. The Copilot SDK adapter
depends on HarnessV1 directly, so it needs a named owner.

8. Test Plan

  1. Phase 1: spike. Confirm with a throwaway script:
    • @ai-sdk/harness and @ai-sdk/mcp run under Bun 1.3.2;
    • Pi with activeTools limited to host tools exposes nothing else;
    • mcpClient.tools({ schemas }) against GitHub's remote MCP server works
      with an App user token, including for an account without Copilot;
    • a host-process Copilot SDK adapter can implement HarnessV1.doStart.
  2. Phase 2: contract suite and fake. Red: contract tests over a fake
    HarnessV1 asserting the active tool set, output, abort, and destroy.
    Green: harness/contract.ts.
  3. Phase 3: Copilot SDK adapter. Red: contract suite against
    createCopilotSdk() with a stubbed Copilot client, including output
    through its internal result tool. Green: move client.ts and runtime.ts
    into the adapter, and add the lint restriction.
  4. Phase 4: Chopin tools on AI SDK. Red: repository tools read the
    repository from toolsContext; a bound GitHub tool called with an injected
    owner/repo still addresses the channel's repository. Green: port
    tools.ts and repository.ts; add harness/github-tools.ts.
  5. Phase 5: Planner on HarnessAgent. Red: chat/service.test.ts drives
    translate with AI SDK stream parts, including the inactive-tool check.
    Green: switch chat to plannerAgent.stream; remove SessionEvent and
    permissions.ts.
  6. Phase 6: workers on structured output. Red: summary and research return
    parsed output. Green: switch both jobs.
  7. Phase 7: deployment modes. Red: a host-login auth fallback is refused on a
    public bind and accepted on loopback. Green: startup check.
  8. Phase 8: Pi adapter. Red: contract suite against @ai-sdk/harness-pi.
    Green: add it to the harness map.
  9. Phase 9: the author's runtime. Red: contract suite against the runtime's
    own HarnessV1 package. Green: add it to the harness map, with its
    classifier model end to end.
  10. Phase 10: E2E harness coverage. Add a fake HarnessV1 and a fake GitHub
    MCP server to e2e, so a scripted Planner turn runs through the real
    projection, tools, and sockets alongside the existing AGENT=off suite.

Existing coverage must stay green: bun test, bun run test:postgres,
bun run e2e, bun run types, bun run ci.

Interactive verification:

  • HARNESS unset: @chopin streams a reply that cites a repository file and a
    pull request from the channel's repository; the log lists only Chopin's tools.
  • HARNESS=pi with an explicit HARNESS_AUTH: the same conversation works.
  • A prompt asking for a pull request in another repository returns data only
    from the channel's repository.
  • HARNESS_AUTH=auto on 0.0.0.0 refuses to start; on 127.0.0.1 it starts.
  • grep -rn "@github/copilot-sdk" apps/server/src matches only the adapter.

9. Open Questions / Unresolved Issues

Resolved:

  • Harness layer: adopt @ai-sdk/harness; the harness owns the loop.
  • Tools: Chopin's document, repository, and GitHub tools are host tools
    bound through toolsContext; no harness built-ins; no snapshot.
  • GitHub access: createMCPClient on the host with a schemas
    allowlist and bindRepository; no installation tokens.
  • GitHub tool allowlist: today's effective set, list_pull_requests and
    pull_request_read with all nine methods; no search tools.
  • Sessions: disposable, destroyed after every turn.
  • Permissions: configuration only; no human approval pauses.
  • Distribution: adapters in the harness map only, each reviewed and
    passing the contract suite.
  • Local model credentials: the harness's own signed-in state; Chopin
    stores none.
  • Shared-key costs: documented, no Chopin quotas in this revision.
  • Second adapter: @ai-sdk/harness-pi, then the author's runtime with
    its classifier.

No open questions remain for this revision. The Phase 1 spike can still
reopen a decision if an assumption fails.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions