commit 31d2568cf13643602ba7bc0dd80bfb086b763bdc from: Isaac Meerleo date: Sat Jul 18 15:47:48 2026 UTC Document the brokered Artifact contract commit - d120f420c9b56088098e2f370bc6e1919ca8564d commit + 31d2568cf13643602ba7bc0dd80bfb086b763bdc blob - d246d795a3bbf6ae26e363738c23e91767f82a41 blob + ca9e3a2eff0c37a2e68c89d29cceba02314cb2a0 --- docs/design/m5-tools.md +++ docs/design/m5-tools.md @@ -90,14 +90,17 @@ clean per-tool error. rename, preserving the existing mode; offered only under `allow_write`. `multi_edit` applies 1–64 pairs to an in-memory copy, all-or-nothing, naming the failing 1-based index. -- **shell** — `/bin/ksh -c` in the project dir, combined output - capped at 256 KiB, 30 s wall clock then process-group kill. +- **shell** — `/bin/ksh -c` in the project dir, combined output captured by + the private Artifact role under the 16 MiB stream and bounded generated- + result policy in behavior.md §5; 30 s wall clock then process-group kill. - **grep** / **find** / **ls** — `grep -RnI`, `find`, `ls -la` - semantics, 256 KiB caps. + semantics under the same Artifact capture policy. - **shell_bg** / **shell_output** / **shell_kill** — a job table; - each job in its own process group, exempt from the 30 s clock, a - 256 KiB ring and bounded drain work per pump; `max_bg_jobs` cap - returns a clean error rather than queueing. A direct-child + each job in its own process group, exempt from the 30 s clock. Each + `shell_output` freezes one immutable Artifact segment and gives a running job + a fresh segment; large results expose only a Session-local opaque id that + `output_read` can page. The `max_bg_jobs` cap returns a clean error rather + than queueing. A direct-child supervisor anchors each command group until the worker reaps it, preserves the command's exact wait status, and watches a worker- liveness pipe, so ordinary group descendants die even if @@ -105,6 +108,12 @@ clean per-tool error. Turn BEGIN/COMMIT/ABORT controls and are killed before an abandoned Turn is reported (I12/I14, best-effort — `setsid(2)` escapes). +The initial M5 baseline used direct 256 KiB result caps and a 256 KiB +background ring. The current Artifact broker supersedes those storage details; +the confinement, process-group, and Turn-transaction mechanics above remain +the M5 foundation. The pinned current interface and limits live in +behavior.md §5. + ### File freshness A per-process table records dev/ino/size/mtime as `read` returns each blob - 93b2ed5c983e10aa82d0f08184258aa273dc1e90 blob + d20c2fdf68454c0e8a3556cfbe3a643ab9c0b659 --- docs/design/m9-subagents.md +++ docs/design/m9-subagents.md @@ -4,7 +4,7 @@ Status: implementation and verification complete, 2026 The OpenBSD `regress/agent` suite passes 155 checks. This slice adds the model-facing `agent` tool, its Owner-bounded route palette, fresh read-only workers, bounded execution, and the lead-only journal boundary required by -behavior.md §5.12 and invariant I15. The combined normal and hardened release +behavior.md §5.13 and invariant I15. The combined normal and hardened release gates also pass as recorded by `docs/design/m10-release.md`. ## Surface and routing blob - d1b228663ae914c56bda016364fe0548959da2a0 blob + 3499d090bca621c22c42b1227432d2af1e749ce4 --- handoff/README.md +++ handoff/README.md @@ -63,7 +63,7 @@ state (HEAD of 2026-06-26, 24 regress suites passing), decisions by the author: strict behavior parity, known-flaw fixes promoted to requirements (invariants.md I13/I14), and a set of deliberate additions beyond parity — the kernel-confined subagent -tool with per-call model routing (behavior.md §5.12, invariants.md +tool with per-call model routing (behavior.md §5.13, invariants.md I15), and a harness feature set chosen after a gap analysis against contemporary agent harnesses: project context files with `#` capture, bracketed paste, external-editor compose, a message queue with blob - 869969cca76c2c94f1622d406a08b66b2ce55efd blob + 1236d3d4636fe01ebfe6c59dd8d2c35cd142ed0d --- handoff/behavior.md +++ handoff/behavior.md @@ -407,6 +407,14 @@ engineering). All file paths are interpreted relative directory; absolute paths within it are fine. Every tool result is bounded; every boundary value obeys invariants.md I13. +The raw stdout/stderr of `shell`, `grep`, `find`, `ls`, `shell_bg`, and +`shell_output` is brokered through the private Artifact role, not a Tool-owned +ring buffer. Each captured stream retains at most 16 MiB. A complete generated +result is inline only when its serialized document fits 256 KiB; otherwise it +contains bounded head/tail previews and an opaque, Session-local output id. +Private paths never cross the role boundary. `output_read` pages that id by raw +byte offset (64 KiB default, 128 KiB maximum), as UTF-8 text or exact base64. + Always offered: 1. **read** `{path, offset?, limit?}` (required: path) — return file @@ -423,37 +431,47 @@ Always offered: request. Line scanning is length-aware end to end (I13). Description steers: read before editing, use `ls` for directories, read only the window needed. -2. **write** `{path, content}` (required: both) — create or fully +2. **output_read** `{id, offset?, limit?, encoding?}` (required: id) — + Lead-only; read one bounded raw-byte page from a Session-local opaque output + id. + `offset` defaults to 0, `limit` defaults to 64 KiB and is capped at + 128 KiB, and `encoding` is `text` (default) or `base64`. Text returns only + a valid UTF-8 prefix and directs the model to retry the same cursor as + base64 when necessary. Results report returned/total bytes, next offset, + EOF, and whether the retained object is pending in the current Turn or + committed and available. Unknown, expired, malformed, and out-of-range ids + are clean generated errors; ids are never paths. +3. **write** `{path, content}` (required: both) — create or fully overwrite; atomic (temp + fsync + rename), preserving an existing file's mode; description steers: prefer `edit` for changes, don't create unasked documentation. Offered only when `allow_write` is on. -3. **edit** `{path, old_string, new_string}` (required: all) — replace +4. **edit** `{path, old_string, new_string}` (required: all) — replace exactly one occurrence; fail if `old_string` is absent or ambiguous; description steers: byte-exact match including indentation, keep the match minimal but unique. Offered only when `allow_write` is on. -4. **shell** `{command}` (required: command) — run via `/bin/ksh -c` - in the project directory; combined stdout+stderr, capped at - 256 KiB with a truncation marker; 30-second wall clock, then kill; +5. **shell** `{command}` (required: command) — run via `/bin/ksh -c` + in the project directory; combined stdout+stderr uses the 16 MiB Artifact + capture and generated-result policy above; 30-second wall clock, then kill; description must state plainly that there is no network access and name the web tools as the alternative, and warn about destructive commands. -5. **grep** `{pattern, path?}` — regex search over file contents +6. **grep** `{pattern, path?}` — regex search over file contents (recursive, line-numbered, binary-skipping, `grep -RnI` - semantics), output capped at 256 KiB. -6. **find** `{path?, name?}` — list files, optional glob filter, - capped at 256 KiB. -7. **ls** `{path?}` — long listing (`ls -la` semantics), capped at - 256 KiB. + semantics), under the same Artifact capture policy. +7. **find** `{path?, name?}` — list files, optional glob filter, under the + same Artifact capture policy. +8. **ls** `{path?}` — long listing (`ls -la` semantics), under the same + Artifact capture policy. Offered while `web_search` is on (default): -8. **web_search** `{query}` (required) — brokered web search returning +9. **web_search** `{query}` (required) — brokered web search returning titles, URLs, snippets (reference: Kagi, 10 results, per-field caps). The search provider sits behind a seam: `search_provider` names the backend, "kagi" is the one that exists. -9. **web_fetch** `{url}` (required) — HTTPS-only fetch returning +10. **web_fetch** `{url}` (required) — HTTPS-only fetch returning readable text (HTML reduced to text); raw body cap 2 MiB, returned text cap 256 KiB with truncation marker; 30-second deadline; does not follow redirects — a redirect status is reported to the model @@ -462,7 +480,7 @@ Offered while `web_search` is on (default): Offered only when `http_allow` is configured: -10. **http_request** `{url, method?, headers?, body?}` (required: +11. **http_request** `{url, method?, headers?, body?}` (required: url) — arbitrary-method HTTPS request with model-supplied headers and body to hosts passing the allow/block patterns (§6); response is the status plus body, non-2xx is a result rather than an @@ -470,11 +488,11 @@ Offered only when `http_allow` is configured: Offered when any model-invocable skill exists: -11. **skill** `{name}` (required) — §4. +12. **skill** `{name}` (required) — §4. New in this revision, offered while `max_subagents` > 0 (default): -12. **agent** `{prompt, label?, model?}` (required: prompt) — spawn a +13. **agent** `{prompt, label?, model?}` (required: prompt) — spawn a subagent: a fresh conversation with the same system conventions as the lead, whose task is the prompt and whose final assistant text is returned as the tool result. Constraints (also invariants.md @@ -520,7 +538,7 @@ New in this revision, offered while `max_subagents` > Offered when `allow_write` is on (with write and edit): -13. **multi_edit** `{path, edits}` (required: both) — `edits` is an +14. **multi_edit** `{path, edits}` (required: both) — `edits` is an array of 1–64 `{old_string, new_string}` objects applied in order to an in-memory copy of the file, each `old_string` required to match exactly once in the intermediate content produced by the @@ -535,7 +553,7 @@ Offered when `allow_write` is on (with write and edit) Offered while `max_bg_jobs` > 0 (default), never to subagents: -14. **shell_bg** `{command}` (required) — start a command under +15. **shell_bg** `{command}` (required) — start a command under exactly `shell`'s confinement and working directory, in its own process group, exempt from the 30-second clock; returns a job id. At the `max_bg_jobs` cap a further call returns a clean error @@ -544,11 +562,12 @@ Offered while `max_bg_jobs` > 0 (default), never to su (builds, test suites); poll with `shell_output` instead of re-running; kill jobs no longer needed; the same no-network warning as `shell` applies. -15. **shell_output** `{id}` (required) — return and consume the job's - accumulated stdout+stderr (bounded ring buffer, 256 KiB per poll, - overflow marked) plus running/exited status and the exit code - once exited. -16. **shell_kill** `{id}` (required) — terminate the job's process +16. **shell_output** `{id}` (required) — atomically freeze and return the + job's accumulated Artifact segment plus running/exited status and the exit + code once exited. A still-running job adopts a fresh 16 MiB segment; the + frozen segment follows the same inline/opaque-id policy above. Repeated + polls therefore advance immutable segments rather than overwriting a ring. +17. **shell_kill** `{id}` (required) — terminate the job's process group. Background jobs are session-scoped: a job spawned by a turn that @@ -634,7 +653,7 @@ Keywords (global or in `match` unless noted). Defaults | `web_search` yes\|no | offer web_search + web_fetch [yes] | | `search_provider` *string* | search backend name [kagi] | | `max_subagents` *number* | concurrent subagent cap; 0 withholds the agent tool [4] | -| `subagent_model` *string* | model id subagents use by default [unset: the lead's active model]; joins the agent tool's per-call palette (§5.12) | +| `subagent_model` *string* | model id subagents use by default [unset: the lead's active model]; joins the agent tool's per-call palette (§5.13) | | `subagent_provider` *string* | name of a configured provider block (or the implicit default) whose route and credential `subagent_model` is resolved against [the lead's active provider] | | `max_bg_jobs` *number* | concurrent background shell jobs; 0 withholds shell_bg/shell_output/shell_kill [4] | | `project_context` yes\|no | select the full or restricted stored System baseline; on a new Session, `yes` admits personal and FUGU.md/AGENTS.md context (§4) [yes] | blob - cffeea1d6d36fc90de905cd182af3d127326fa3d blob + 6886a6768a261f58877ecf9f00193d5d024da67b --- handoff/verification.md +++ handoff/verification.md @@ -167,7 +167,10 @@ states the property, not just examples: file byte-identical and names the index. - **Background jobs:** `max_bg_jobs` honored; jobs die on turn failure, cancellation, session end, and drawer switch; unknown and - prior-session ids error cleanly; ring-buffer overflow marked. + prior-session ids error cleanly. Repeated `shell_output` polls freeze exact + immutable Artifact segments, preserve unavailable reservations, and retire + available/unavailable spools through exact DROP/ACK, RELEASE, and job-wide + CANCEL/ACK races without reusing historical ownership. - **Retry policy:** transport retries never occur after the first content delta; `Retry-After` clamped to [1, 60] and parsed as untrusted (I13); cancellation during backoff leaves the journal clean. Separately, exact @@ -185,6 +188,16 @@ states the property, not just examples: - **Read paging:** offset/limit windows respect the byte cap against a newline-free 1 MiB line; totals and next-offset reported; zero and negative values rejected. +- **Artifact output paging and lifetime:** `artifact_result` computes the + complete serialized 256 KiB boundary and its +1 transition for printable, + escape-heavy, NUL, and invalid-UTF-8/base64 shapes. `artifact_page` pins raw + byte cursors, UTF-8 prefix behavior, base64 fidelity, and the 128 KiB page + cap. `artifact_store` covers pending/available/expired reads, 16 MiB stream, + 128 MiB Session and 1 GiB global quotas, reconciliation, eviction, and + transaction rollback. `turn` drives a real opaque id through Provider + `output_read`, Coordinator parsing/defaults, the Artifact role, an exact + NUL/invalid-byte base64 page, and the next Generation; a delayed response is + cancelled to prove its exact DATA/READ_RESULT drain precedes rollback. - **Edit-diff display:** the display cap and elision marker exercised; binary and over-cap content degrades to a summary; hostile diff bytes are length-checked at the role boundary (I13); blob - 6bdd16c3ea6e1337b342f6766412b96619042b51 blob + 05206b69308fbd2003a5eb6529024c25c592cad8 --- regress/README +++ regress/README @@ -56,7 +56,12 @@ I10 mutual distrust across roles imsgev (partially con (a hostile second terminal cannot flip a committed turn; a tool subprocess cannot reach the - coordinator control fd); sandbox + coordinator control fd; normal exact + Artifact rotation/retirement and delayed + read frames cross the live role mesh); + tools (hostile Artifact rotation/drop/cancel + mechanics); artifact_store, artifact_page, + artifact_lock; sandbox (fdguard: closefrom closes the inherited control fd) I11 least linkage deploy (installed ELF NEEDED sets and @@ -74,7 +79,11 @@ I13 length-aware strings buf, json (NUL round-trip), sse (NUL transparency), anthropic, openai, http, web, output (NUL-bearing framing, headers, bodies, - and JSON emission); boundary-wide + and JSON emission); artifact_result + (whole-document printable/escape/NUL/base64 + boundaries), artifact_page (raw-byte text and + base64 cursors), turn (binary output_read); + boundary-wide property grows with each subsystem I14 resume fidelity journal (transactional replay: @@ -132,6 +141,10 @@ tools (behavior.md 5) agentcfg (agent schema and rou tools (executors, caps, freshness, NUL rejection, bounded background draining, supervisor/liveness cleanup); + artifact_result, artifact_page, artifact_store, + artifact_lock (brokered output retention and + paging); turn (opaque-id output_read loop and + cancelled exact-response drain); web (search/fetch/request policy, caps, credential scoping); turn (full file loop); agent (ephemeral- @@ -209,7 +222,15 @@ src/common/openai_req.c, src/fugu/parse.y, conf.c conf src/fugu/agentcfg.c agentcfg; agent (live schema/route) src/fugu/priv.c, coord.c sandbox (spawn/handshake/teardown), - turn, agent (lead and subagent turn loops) + turn (Lead loop and Coordinator Artifact + paging seam), agent (subagent turn loop) +src/fugu/artifact_result.c, + artifact_capture.c artifact_result; tools; turn +src/fugu/artifact_page.c artifact_page; turn (real output_read) +src/fugu/artifact_lock.c, + artifact_scope.c, + artifact_pipe.c artifact_lock; sandbox; turn +src/fugu-artifact (main.c, store.c) artifact_store; sandbox; turn src/fugu/journal.c journal, turn src/fugu/turn_txn.c turn_txn (pure ordering plus real-Journal integration); turn (cache/SIGINT path) blob - 7e0fa7b950d679d9a1e25eed508a42d5c3d584fd blob + 3fc4f883855bf98a75d838f31a2dca6f737e4521 --- src/fugu/fugu.1 +++ src/fugu/fugu.1 @@ -659,6 +659,16 @@ Committed retained output survives resume, and Compaction until quota cleanup evicts it. Retained output can contain credentials or other secrets printed by a command; Owners should treat the entire managed root as sensitive data. +At startup, +.Nm +may create or validate the owner-only +.Pa ~/.fugu +directory before resolving this root. +If its physical path overlaps the project tree or another persistent Lead Tool +filesystem authority, +.Nm +refuses the invocation before spawning workers, opening or changing a Journal, +or contacting a Provider. . .Sh ENVIRONMENT .Bl -tag -width FUGU_CONF