commit 131c40ba416d22aaef138deb326e9426e4747f5e from: Isaac Meerleo date: Tue Jul 14 12:18:21 2026 UTC integrate subagents and finish release hardening Add bounded read-only subagents with per-call model routing, concurrent provider request handling, transport and credential hardening, expanded regression coverage, and automated live release checks. commit - 0016fef86afa0f5970f280391a9d6ce5e18526ff commit + 131c40ba416d22aaef138deb326e9426e4747f5e blob - 9dd88637e89c14cb90eb62cee63bccffc86ee09a blob + ba088d2ecc6c709a83f441056a1a7bfab618a34f --- CONTEXT.md +++ CONTEXT.md @@ -127,7 +127,9 @@ unknown. iteration-capped) turns survive in the **Journal**'s replay. - A **Subagent** runs its turns through the same **fugu-api** custodian but executes tools only in the read-only **fugu-tool** - role. + role. Correlated model connections make progress concurrently inside + that one custodian; the Coordinator queues brokered web calls and + admits them to **fugu-web** one complete request at a time. - A **Steer** lives inside its **Turn**; a **Queue** entry lives outside any turn. blob - 278414b2ba029c3361bb483193d3da1b72279338 blob + bd7f5faab24a7828ea343f170255ed815be713c2 --- Makefile +++ Makefile @@ -6,7 +6,8 @@ SUBDIR= src # The regress directory exists as a file, so the target must be # phony or make calls it up to date. -.PHONY: regress obj cleandir check +.PHONY: regress obj cleandir check live-check live-check-privileged \ + live-check-help # Run every suite (charter.md). regress: @@ -24,4 +25,15 @@ check: cd ${.CURDIR} && ${MAKE} cd ${.CURDIR} && ${MAKE} regress +# Deliberately outside `check`: these spend real quota/reach the public +# network, while the privileged half installs a package on a clean host. +live-check: + cd ${.CURDIR} && exec ./scripts/live-check credentialed + +live-check-privileged: + cd ${.CURDIR} && exec ./scripts/live-check privileged + +live-check-help: + @cd ${.CURDIR} && ./scripts/live-check help + .include blob - 18b2c9e40c9d725537ebc25f9351e0f62e93e6de blob + b6724082c80a911256d6058e796d6d9fd292ef01 --- README +++ README @@ -3,8 +3,9 @@ fugu — an OpenBSD-native AI coding agent fugu is an interactive terminal program that pairs a large-language- model chat API with local tools (file read/write/edit, shell, search, -brokered web access) so a model can work inside a project under the -user's direction. It is privilege-separated in the manner of OpenBSD +brokered web access, and bounded read-only subagents) so a model can +work inside a project under the user's direction. It is +privilege-separated in the manner of OpenBSD base daemons: every security property is enforced by pledge(2), unveil(2), and process boundaries, never by prompt or convention. @@ -35,6 +36,16 @@ A hardened debug build compiles with UBSan in trap mod make FUGU_DEBUG=1 check +Credentialed/public-network and clean-host package checks are explicit release +targets, never part of the hermetic gate: + + make live-check-help + make live-check + make live-check-privileged + +The required environment and safety prerequisites are documented in +`docs/release-live-checks.md`. + The checks selected by `-fsanitize=undefined` remain enabled except Clang's function-type probe: that probe reads metadata from executable text and is incompatible with OpenBSD's execute-only PIE mappings. Indirect-call types blob - 5607a0f5a77911e5fdefdcfbbdb3dca3a7f2c668 blob + 4be9595a6d036987412ca2436d75674113b078ea --- docs/adr/0002-libevent-over-poll.md +++ docs/adr/0002-libevent-over-poll.md @@ -5,9 +5,12 @@ poll(2) loops and libevent. We chose libevent for ever a dispatch loop: it is the shape of the cited privsep exemplars (vmd's `imsg_event_add` discipline transfers verbatim), and evtimer covers fugu's dense timer needs — retry backoff with jitter, shell -wall-clocks, subagent multiplexing in fugu-api — without hand-rolled +wall-clocks, and correlated subagent dispatch — without hand-rolled deadline bookkeeping. The known cost is callback-driven control flow -in the turn state machine. +in the turn state machine. In fugu-api, each correlation id owns its DNS, +connect, TLS, HTTP, retry-timer, and provider-stream state. Requests therefore +make concurrent progress while one process remains the sole credential +custodian. ## Spec conflict resolved here blob - 55584aaa26f6e6f96dd1b27085b7996c7dfe9bfe blob + b72b639bd6ac77e55ca61797dbf54ab77a418032 --- docs/design/m10-release.md +++ docs/design/m10-release.md @@ -1,8 +1,9 @@ # M10 provider and release completion -Status: implementation verified, 2026-07-10. The exact source snapshot -passes clean-room normal and `FUGU_DEBUG=1` checks on OpenBSD; the -credentialed and privileged operator checks remain outstanding as documented +Status: implementation and integrated hermetic verification complete, +2026-07-11. Clean-room normal and `FUGU_DEBUG=1` gates both pass all 28 +OpenBSD suites with no warnings, sanitizer diagnostics, or failures. +Credentialed and privileged operator checks remain outstanding as documented in `docs/release-live-checks.md`. There is no normative document that assigns a complete feature bundle to @@ -23,11 +24,10 @@ M10 contains: otherwise advertise inaccurately; 5. linkage, package, clean-build, hardened-build, and documented live checks. -M10 does not implement the M9 `agent` tool or I15 subagent orchestration, and -does not take ownership of concurrent M7 or M8 feature work. The release as a -whole cannot be called complete until M9 lands because `fugu(1)` must describe -the agent tool and the full invariant table must have no M9 hole. M10 may be -built and verified independently before that integration. +M10 was built and verified independently of the M9 `agent` tool and I15 +subagent orchestration. M9 is now integrated as the separate slice described +in `docs/design/m9-subagents.md`; release documentation and traceability cover +the combined surface. ## OpenAI protocol @@ -64,6 +64,16 @@ Anthropic providers retain `x-api-key`. The same TLS- policy, provider-error surfacing, endpoint confinement, and credential custody apply to both protocols. +The custodian runs correlated requests as independent libevent state machines +behind a bounded live-connection pool. Request bodies have an aggregate +memory ceiling, response reads have a callback budget and IPC high/low-water +pause, and every phase uses monotonic deadlines (including a five-second total +error-body deadline). Retryable 429/5xx status remains retryable even when its +error body is truncated or malformed. Provider errors are checked against +the complete configured credentials before their user-visible form is +truncated; authentication headers, wire buffers, and reflected error buffers +are wiped when released. + ## Release-facing invocation closure The release gate checks the command-line interface as installed, not merely @@ -145,9 +155,10 @@ secrets, provider blocks, endpoint defaults, every key tools, protected paths, and the set-group-id environment-override rule. Both manuals use mdoc(7) and must pass `mandoc -Tlint`. -The M9 agent surface is documented during release integration, once its actual -UI and failure behavior exist; prose must not claim an unavailable execution -path merely to make M10 appear closed. +The integrated M9 agent surface is documented from its actual UI, routing, +confinement, and failure behavior. In particular, the manual records bounded +worker waves and correlated concurrent model connections inside the one API +credential custodian. ## Verification @@ -162,6 +173,10 @@ responses, malformed and oversized append-only journal SIGINT races, Turn-scoped background rollback, killed/stopped supervisors, forced executor death, bounded continuous background output, and complete subprocess tail/status reporting. +The API reflection regression includes credentials longer than the error-frame +limit in both SSE and non-2xx JSON errors, and transport coverage includes a +truncated retryable 503 and a slow-drip body that must hit the independent +five-second aggregate deadline. Release checks cover: @@ -169,10 +184,10 @@ Release checks cover: - a `DESTDIR` install with exact paths, modes, and ownership metadata; - port `makesum`, `make fake`, `make package`, plist, and lib-dependency checks; - normal and `FUGU_DEBUG=1` clean-room `make check` runs on OpenBSD 7.9; -- a documented manual checklist for a real default-provider turn, live - `web_fetch`, expired-certificate error surfacing, and a clean-system package - install reading root-owned group-readable `/etc/fugu.conf` through the - set-group-id front binary. +- a documented manual checklist for a real default-provider turn, a routed + subagent, live `web_fetch`, expired-certificate error surfacing, and a + clean-system package install reading root-owned group-readable + `/etc/fugu.conf` through the set-group-id front binary. Credentialed and privileged live checks are never silently replaced by a stub. If they are not run in the development environment, the handoff names blob - a7a1fd398ad34974868df7b55d7a2c32774ff52b blob + 7fcbb3a23b7e972c5f3673034636095c72de2c90 --- docs/design/m3-privsep.md +++ docs/design/m3-privsep.md @@ -64,7 +64,7 @@ re-pledging without the wider promise and locking unve | **fugu-api** | `stdio rpath inet dns` to load CA bundle | `stdio inet dns` | locked (`unveil(NULL,NULL)`) — cannot open any file | exec; write/open files (I1); reach anything but TLS to the endpoint | | **fugu-web** | `stdio rpath inet dns` to load CA bundle | `stdio inet dns` | locked | exec; write/open files; reach tty (I7) | | **fugu-tool** | `stdio rpath ... unveil` to unveil the tree | `stdio rpath wpath cpath proc exec` (execpromises: **no inet/dns**) | project tree **rwxc** minus `protect` paths; `/usr /bin /sbin /lib /libexec` rx for subprocesses; **not** `/etc` (I5 "nothing else") | reach network from tools or their children (I4, execpromises survive exec) | -| fugu-tool `-r` (subagent) | same | `stdio rpath proc exec` (no wpath/cpath; execpromises no inet) | project tree **rx** only (minus protect); system rx | write/create anything (I15); network even with allow_subprocess_net | +| fugu-tool `-r` (subagent) | same | `stdio rpath proc exec` (no wpath/cpath; execpromises retain ksh startup write/fattr classes but no inet/dns/unveil) | project tree **rx** only (minus protect); system rx | write/create anything (I15; locked unveil is the subprocess wall); network even with allow_subprocess_net | The tree carries `x` so the shell tool can run project-local scripts (`./configure`, `./build.sh`); the read-only subagent keeps `rx` (run @@ -78,7 +78,9 @@ OpenBSD 7.9 (the openbsd-refs cite an older tree); tem met with `wpath cpath` plus an unveil of the temp directory. Config-gated variants: -- `allow_write no` → fugu-tool drops `wpath cpath` (I6), tree unveiled `r`. +- `allow_write no` → fugu-tool drops `wpath cpath` (I6), tree unveiled `rx`; + subprocess exec promises may still name write classes needed by ksh startup, + but the locked veil grants no writable path. - `allow_subprocess_net yes` → fugu-tool's **execpromises** gain `inet dns` and the resolver/TLS-trust paths are unveiled `r`; the tool role's own pledge is unchanged (widens only the subprocess blob - cc3ba3a98d00ec1bf044b47920999f5265855388 blob + 4bf8c11f404c33c0431efa925ed9dcb13de14855 --- docs/design/m5-tools.md +++ docs/design/m5-tools.md @@ -15,7 +15,7 @@ breakpoint on the last tool. The stream decoder alread `ASTREAM_TOOL_BEGIN` / `ASTREAM_TOOL_INPUT`, relayed to the coordinator as `A_TOOL_BEGIN` / `A_TOOL_INPUT`. `fugu-tool` already unveils the tree and pledges without `inet`/`dns`, with an -execpromises ceiling that survives `exec` (I4), and drops +execpromises ceiling that survives `exec` (I4). Its own steady pledge drops `wpath cpath` in the read-only (`-r`, subagent) role. M5 adds the tool *surface*, the *executors*, the *run protocol*, and @@ -41,10 +41,13 @@ are ordinary tools whose executor manages a job table; does not special-case them. `hello_tool` carries the two gates so the confined room can set its -own pledge and unveil; the `protect` globs follow as NUL-separated -trailing bytes and are unveiled with empty permissions before any -untrusted input is touched (I5/I9). The working directory is -inherited through `fork`, so no path is sent. +own pledge and unveil. The coordinator expands `protect` once and +stores the NUL-separated literal snapshot in a read-only unlinked +file inherited by tool roles; this avoids the imsg payload ceiling. +The worker validates and `pread`s the snapshot, unveiling every path +with empty permissions before any untrusted input is touched (I5/I9). +The working directory is inherited through `fork`, so no cwd path is +sent. ### Why the subprocess pledge is broad @@ -58,8 +61,13 @@ getpw proc exec prot_exec tty` — and pointedly exclu and `unveil`. Excluding `unveil` (and never granting it in the custodian's own final pledge) locks the veil, so no subprocess can re-widen the tree it was confined to. The read-only (subagent) ceiling -drops the write promises. `allow_subprocess_net` appends `inet dns` to -the execpromises only — never to the custodian role itself. +keeps the write and file-attribute exec promises because OpenBSD +`/bin/ksh` requests them while establishing its startup pledge. The +tool role's own steady pledge drops `wpath`/`cpath`, and the inherited, +locked project unveil is `rx`, so those subprocess promise classes grant +no effective write/create authority. For the lead executor, +`allow_subprocess_net` appends `inet dns` to the execpromises only — never to +the custodian role itself. The subagent role forces that gate off. ## The executors (fugu-tool/tools.c) blob - /dev/null blob + 7d9db4c7b455643ddce1749314ba82031f979427 (mode 644) --- /dev/null +++ docs/design/m9-subagents.md @@ -0,0 +1,155 @@ +# M9 — ephemeral read-only subagents + +Status: implementation and verification complete, 2026-07-11. +The OpenBSD `regress/turn` agent target passes 126 checks. This slice adds the +model-facing `agent` tool, its operator-bounded route palette, fresh read-only +workers, bounded execution, and the lead-only journal boundary required by +behavior.md §5.12 and invariant I15. The combined normal and hardened release +gates also pass as recorded by `docs/design/m10-release.md`. + +## Surface and routing + +The lead receives `agent {prompt, label?, model?}` when `max_subagents` is +non-zero. `prompt` is required. `label` is optional presentation metadata; +the coordinator otherwise assigns `agent-N`. `model`, when present, must +exactly match an enum value in the dynamically built schema. Bad types, +oversized identifiers, malformed JSON, and out-of-palette choices become tool +errors before any worker or provider request is made. +The prompt, label, and final report are capped at 4 MiB, 63 bytes, and 1 MiB, +respectively. Labels are limited to ASCII letters, digits, underscore, +hyphen, and dot before they can enter terminal progress output. + +The configured model strings are snapshotted at startup: the implicit +provider's model, every named provider block's model, and `subagent_model`. +When no subagent provider override exists, rebuilding the schema binds the +snapshotted `subagent_model` to the lead's then-active provider. Equal model +strings on different providers remain distinct routes; identical routes are +deduplicated. With only the implicit provider, model identifiers are bare. +Once named providers exist, the implicit route is `default:model`, named +routes are `name:model`, and a block literally named `default` uses +`@default:model`. The complete enum string is the lookup key, so model ids +may themselves contain colons without an ambiguous split. +An implicit provider with no effective key is omitted from the palette and +cannot be selected. + +When a call omits `model`, `subagent_provider` and `subagent_model` override the +lead's live route independently. Thus either setting may be absent and fall +back to the corresponding active lead value. A later `/model` selection can +affect this default for a future call, but cannot reroute a worker already +started. Endpoints and credentials never enter the palette snapshot: the +resolved provider slot is sent to the existing `fugu-api` custodian. +In configuration, `subagent_provider "@default"` explicitly names the implicit +slot, while `subagent_provider "default"` still names a provider block literally +called `default`; this is the inverse of their two unambiguous palette prefixes. + +## Process and authority boundary + +Each accepted call fork-execs a fresh `fugu-tool -r`. The coordinator sends it +the selected provider protocol and model, current system/personal/project +context, the self-contained prompt, tool gates, and the exact startup-expanded +protect-path snapshot. It sends no lead messages, compaction summary, journal, +provider key, or web token. The worker constructs its own canonical +conversation and asks the coordinator to broker provider and web traffic. +The spawn path pins only the new control channel and applies `closefrom`, so +the already-open session journal and other worker channels do not survive the +exec. + +The agent role fixes the kernel ceiling before consuming its prompt: the +project is unveiled `rx`, the role's own steady pledge omits write/create and +network promises, and background capacity is zero. Subprocess exec promises +still include the write and file-attribute classes that OpenBSD `/bin/ksh` +needs while establishing its startup pledge. They omit `inet`, `dns`, and +`unveil`; more importantly, they inherit the already-locked project `rx` +unveil, which remains the effective kernel wall against every write/create +attempt. This stays true even when the lead has `allow_write yes` or +`allow_subprocess_net yes`. The local tool definitions are exactly `read`, +`grep`, `find`, `ls`, and `shell`. +`web_search`/`web_fetch` and allowlisted `http_request` are added only under the +same operator gates as the lead and remain brokered through `fugu-web`. +Mutation, background jobs, skills, and `agent` itself are absent; an unexpected +request for any of them is rejected as unavailable as a second line of +defense. + +Protect globs are expanded once by the coordinator before any worker starts; +only `GLOB_NOMATCH` is an ordinary empty match and every other glob error fails +startup closed. The deduplicated NUL-separated literals are materialized in +an unlinked file, reopened read-only, and inherited at a fixed descriptor by +the persistent executor and every dynamic agent. Receivers validate its type +and length and consume it with `pread`, so the snapshot has no imsg-size limit +and concurrent workers cannot disturb a shared offset. Protected contents +remain kernel-unreachable, names may be visible in listings, and a later-created +path that would have matched a glob is not retroactively covered. + +## Bounds, waves, and cancellation + +All agent calls from one provider response are collected before ordinary tools. +They run in result-stable waves no larger than `max_subagents`; excess calls +wait for the next wave. The ephemeral read-only processes in a wave coexist +and make local progress independently. Inside the single credential +custodian, every correlation id owns an event-driven DNS, connect, TLS, HTTP, +retry, and provider-stream state machine. Provider connections therefore +overlap without copying credentials into a coordinator or child. The broker +admits at most eight live hostile streams, queues the remainder, caps aggregate +request memory, yields after bounded reads, and pauses provider reads while its +IPC output is backed up. Absolute connect/write/header, response-total, and +error-body deadlines prevent WANT cycles or slow-drip failures from extending a +request forever. The coordinator admits exactly one complete subagent web +request to the separate web custodian at a time, bounds aggregate queued +arguments, and does not release the next request until the prior terminal +arrives. + +A subagent gets at most 25 tool rounds. Each generation uses the session's +`max_tokens`. After round 25 the worker appends explicit conclusion guidance +and makes one tool-free generation, then marks the report truncated. Any +provider `max_tokens`/`length` stop follows the same one-conclusion path: +partial text is retained, dangling tool calls receive errors without executing, +and the final report is marked. When retained prose exceeds the report cap, +the bounded result preserves both a useful prefix and the final conclusion or +terminal diagnostic. Prompt, assistant text, provider tool-input +bytes, and every emitted tool result/error share the 4 MiB per-agent aggregate. +Request, response, per-tool argument/result, and final-report bounds retain +their narrower protocol limits. + +A print-mode SIGINT, quitting, or choosing another conversation while a wave +is active terminates every dynamic worker and skips queued waves. The +read-only worker's existing SIGTERM path also terminates its current shell +process group. As with every abandoned lead turn, no partial conversation +state commits. Teardown signals the entire wave first and gives all children +one shared one-second grace deadline before killing stragglers, so cancellation +latency does not scale with the wave size. + +## Presentation and persistence + +Workers report `started`, provider request/retry, tool, and completion events. +The coordinator prefixes each with `agent[label]`; curses renders them as +notices and line/print modes write them to standard error. Provider usage from +the workers contributes to the parent turn's aggregate usage. + +Only the final bounded report crosses back as the `agent` tool result. The +subagent conversation, its intermediate assistant text, and its local tool and +web exchanges never enter the session journal. The lead's spawn arguments and +final result are journaled transactionally as an ordinary tool exchange, so a +successful resume has enough information to reconstruct the lead projection +without retaining an ephemeral child transcript. + +## Verification + +`regress/conf` pins named and `@default` provider selection and rejects an +unusable implicit slot. `regress/agentcfg` pins schema shape, exact palette +keys, provider/model fallback independence, duplicate handling, and +invalid-route refusal. +The 126-check end-to-end agent target drives the real worker mesh through a +fresh child tool loop, startup-protected reads, an attempted shell write, final +report, and lead synthesis. It pins the exact read-only tool surface, denied +network and write attempts despite lead opt-ins, a protect snapshot larger than +16 KiB, live progress, lead-only journal boundary, hostile label rejection, and +invalid-route refusal before a child request. Separate scenarios prove cap-1 +waves, cap-2 provider overlap with reverse completion and stable result order, +pool saturation proving retry backoff releases one of eight network slots, +serialized web calls from concurrent children, escaped future literal protect +paths, text-only and dangling-tool provider caps, conclusion and terminal-error +tail retention beyond 1 MiB, tool-input/result aggregate bounds, the 25-round +tool-free conclusion, and SIGINT reaping an active child process group while +abandoning the journal turn. The sandbox suite independently pins +the `fugu-tool -r` role's own `wpath` ceiling and the post-exec no-network +ceiling; the tool suite pins the shared protect and filesystem walls. blob - c832c3716d6686a9f70248e8d59dfe4ca2e30930 blob + f7451d23ab73188d612b7c61ca5af2b66d93cc2b --- docs/release-live-checks.md +++ docs/release-live-checks.md @@ -1,61 +1,104 @@ # M10 live release checks -Status: outstanding operator checks. The hermetic normal and hardened gates -passed on 2026-07-10; the checks below require real credentials, public -network access, or a clean privileged package-install host. +Status: automated operator checks, not yet run against the release package. +The integrated hermetic normal and hardened gates passed on 2026-07-11 as +recorded in `docs/design/m10-release.md`. These checks remain outside +`make check` because they spend real provider/search quota, reach the public +network, or install a package as root. -Run these checks on a clean OpenBSD 7.9 machine after the hermetic normal and -hardened regression gates. They exercise credentials, public DNS/TLS, and -package installation, so regress intentionally does not automate them. Keep -the command transcript, but never record a key or unredacted configuration. +The implementation is `scripts/live-check`. Its two entry points must be run +separately: credentialed model execution must not run as root, while the clean +package test must. -## Default-provider turn +## Credentialed and public-network target -1. Install the package and prepare `/etc/fugu.conf` from the package sample. - Put the real default-provider key in a separate root-owned file. Both files - must be owned by `root:_fugu`, mode 0640. -2. Confirm `fugu -n` reports `anthropic`, `claude-sonnet-4-6`, and a redacted - key. -3. Run `fugu -p 'Reply with exactly: live provider ok'`. -4. Require exit status 0, the requested text on stdout, and no credential in - stdout, stderr, syslog, or the new session journal. +Prepare the installed `/etc/fugu.conf` and its key files as follows: -## Live brokered fetch +- the active provider and model are the expected release defaults (override + `FUGU_LIVE_EXPECT_PROVIDER` or `FUGU_LIVE_EXPECT_MODEL` when intentionally + testing another route); +- `max_subagents` is at least 2, and `subagent_provider` names a real provider + block with `subagent_model` set; +- `web_search yes` and a real Kagi key are configured; +- provider and Kagi credentials live in separate key files, not inline. -1. Configure a real Kagi token in `/etc/fugu.conf` or a secrecy-checked key - file and leave `web_search yes` enabled. -2. Run a turn that explicitly asks the model to use `web_fetch` on a stable - public HTTPS page and report its title. -3. Require the tool to return public page text, the provider to receive the - bounded tool result, and the turn to finish normally. Confirm that no Kagi - or model credential appears in the journal. +As an ordinary user, run: -## Expired-certificate error +```sh +env FUGU_LIVE_SECRET_FILES='/etc/fugu.key /etc/fugu-work.key /etc/fugu-kagi.key' \ + make live-check +``` -1. Ask a credentialed turn to use `web_fetch` on - `https://expired.badssl.com/`. -2. Require the tool result to contain the real libtls certificate-expiry - reason. A generic network error, a successful body, or redirect following - is a failure. -3. Repeat against a provider endpoint with an expired test certificate when - one is available, and require the provider turn to surface the same class - of libtls error after the documented retry policy. +The key-file list is mandatory. Each file must contain exactly one nonempty +printable key line, which the target uses as a fixed pattern when checking +stdout, stderr, the isolated session journals, and a uniquely delimited syslog +slice. It never places a key in an argument or prints one. If the ordinary +user cannot read a key or the system log, the target invokes `doas` for the +comparison. `FUGU_LIVE_DOAS` and `FUGU_LIVE_SYSLOG` can override those +commands/paths. -## Clean package and set-group-ID configuration +The target creates an isolated temporary HOME and project, then proves: -1. On a machine without a pre-existing `_fugu` group, install the generated - package with `pkg_add`. Require group `_fugu` with registered GID 992, - `/usr/local/bin/fugu` owned by `root:_fugu` and mode 2555, ordinary - mode-0555 workers below `/usr/local/libexec/fugu/`, both manuals below - `/usr/local/man/`, and `/etc/fugu.conf.sample` mode 0640 and group `_fugu`. -2. Copy the sample to `/etc/fugu.conf`, add a working key, and preserve - `root:_fugu` mode 0640. As an ordinary user who is not in `_fugu`, run - `fugu -n`; require successful parsing and redacted output through the - executable's set-group-ID read. -3. Create a different user-owned configuration and set `FUGU_CONF` to it. - Run the installed `fugu -n` and require the root-administered - `/etc/fugu.conf` values instead. This proves the privileged execution - ignores development overrides. -4. Replace `/etc/fugu.conf` temporarily with a world-readable or - group-writable copy and require startup to fail before UI or network - activity. Restore the secure file before continuing. +1. `fugu -n` reports the expected active route, redacted keys, the named + subagent route, and enabled web access. +2. A real default-provider turn returns the exact requested marker. +3. Two labelled agents inspect different files in one wave, expose progress, + and return bounded reports through the configured named provider. The lead + calls/reports enter the Journal; child system text and local read results do + not. +4. Real Kagi `web_search` and public `web_fetch` calls return non-error broker + results. +5. `web_fetch` of `https://expired.badssl.com/` returns a genuine certificate + error rather than a generic failure or body. +6. No configured credential appears in terminal output, journals, or syslog. + +An expired provider endpoint is environment-specific. When one is available, +set both `FUGU_LIVE_EXPIRED_PROVIDER_CONF` and +`FUGU_LIVE_EXPIRED_PROVIDER_BIN`; the latter must be an explicit non-setgid +test binary because an installed binary correctly ignores the alternate +configuration. Without them, that single check is reported explicitly as +skipped. + +Development builds can be selected with `FUGU_LIVE_BIN`, `FUGU_LIVE_CONF`, +and `FUGU_LIVE_LIBEXEC`. Run `make live-check-help` for the complete interface. + +## Clean package and set-group-ID target + +Use a disposable, clean OpenBSD 7.9 host where `_fugu`, the installed binary, +and `/etc/fugu.conf` do not already exist. Prepare a self-contained, +root-owned mode-0600 configuration source that uses only key-file directives; +inline secret directives, `include`, and macros are rejected so the audit +cannot hide or miss another secret-bearing directive. Put its root-owned +mode-0600 key files at their final paths; the target changes them to +`root:_fugu` mode 0640 after the package creates the group. Choose an ordinary +test user who is not a member of `_fugu`, then run as root: + +```sh +doas env FUGU_LIVE_PACKAGE=/tmp/fugu-0.1.tgz \ + FUGU_LIVE_CONF_SOURCE=/root/fugu-live.conf \ + FUGU_LIVE_SECRET_FILES='/etc/fugu.key /etc/fugu-work.key /etc/fugu-kagi.key' \ + FUGU_LIVE_CONFIRM="$(hostname):$(sha256 -q /tmp/fugu-0.1.tgz)" \ + FUGU_LIVE_USER=alice make live-check-privileged +``` + +This target intentionally mutates the clean host. It installs the package and +leaves it installed. It verifies: + +1. package creation of `_fugu` with GID 992; +2. the root/group/mode contract for the set-group-ID front binary, all five + workers, both manuals, and `/etc/fugu.conf.sample`; +3. installation of every key and the supplied configuration as `root:_fugu` + mode 0640, while the ordinary user cannot read them directly; +4. successful redacted `fugu -n` execution by the ordinary non-`_fugu` user; +5. set-group-ID execution ignoring that user's `FUGU_CONF` override; +6. fail-closed startup for world-readable and group-writable credential-free + decoy configurations. The real credential-bearing configuration is never + made insecure and is restored atomically even on interruption. + +`FUGU_LIVE_CONFIRM` binds the destructive invocation to the current hostname +and the SHA-256 digest of the package snapshot. A mismatch stops before +`pkg_add`. + +Both targets emit only TAP-like pass/fail/skip lines and a final count. Capture +that output as the release transcript; a nonzero exit means the release check +did not pass. blob - 76a9e1acb693ed240ac510300ffffde2df7b4d27 blob + 504d7b09ea29a1e9cc6f6b8a0403d6e817a06d57 --- etc/fugu.conf.sample +++ etc/fugu.conf.sample @@ -43,7 +43,8 @@ # Allow shell subprocesses to reach the network (the tools themselves # never can). Enabling this permits a steered model to exfiltrate -# readable files; that is the operator's trade. +# readable files; that is the operator's trade. Read-only subagent +# subprocesses remain networkless regardless. #allow_subprocess_net no # Offer web_search and web_fetch, and which search backend to use. @@ -52,9 +53,13 @@ #kagi_token "..." #kagi_token_file "/etc/fugu-kagi.key" -# Reserved for the M9 subagent integration: concurrent cap and the -# model/provider subagents will use by default. The current build parses -# these settings but does not yet offer the agent tool. +# Offer the read-only agent tool and set its concurrent worker cap. +# Multiple calls run in waves of this size; 0 withholds the tool. A +# subagent defaults independently to these model/provider overrides or, +# when either is unset, to the lead's active value. subagent_provider +# names a configured provider block; "@default" explicitly selects the +# implicit provider and requires its API key. A named block literally +# called "default" is selected as "default", without the @. #max_subagents 4 #subagent_model "claude-haiku-4-5" #subagent_provider "work" @@ -68,7 +73,9 @@ #project_context yes # Paths (or glob patterns) relative to the working directory whose -# contents no tool, subprocess, or subagent can reach. +# contents no tool, subprocess, or subagent can reach. The set is a +# startup snapshot: absent literal paths remain blocked if later created, +# while an unmatched glob does not cover later-created matches. #protect ".env secrets/*" # Terminal bell when a turn completes (curses interface only). @@ -95,6 +102,12 @@ # api_key_file "/etc/fugu-work.key" #} +# When named providers exist, the agent tool's model palette qualifies +# routes as "default:model" for the implicit provider and "name:model" +# for a named block. A block literally named "default" appears as +# "@default:model". With no named providers the model id is bare. A +# credentialless implicit provider is omitted from the palette. + # Per-user and per-group overrides: the first matching block wins, # later blocks are not consulted. No secrets in match blocks. #match user "alice" { blob - 46c095c42ad789d8209ecbdeff4ee4dc6d3c9076 blob + c9fa034eef90c2423ff5db5b2f98fa0a74797071 --- port/productivity/fugu/pkg/DESCR +++ port/productivity/fugu/pkg/DESCR @@ -1,13 +1,16 @@ fugu is an OpenBSD-native AI coding agent: a small, privilege-separated coding assistant written in C and sandboxed with pledge(2) and unveil(2). It pairs Anthropic Messages or an OpenAI-compatible endpoint with bounded -local file and shell tools and brokered web search, fetch, and HTTP tools. +local file and shell tools, brokered web search, fetch, and HTTP tools, and +ephemeral read-only subagents. Six executables have disjoint privileges. The terminal role has no network or filesystem access; the API role holds the model credential but cannot open files, write, or execute programs; the web role cannot write or execute; and model-selected local tools cannot reach outside the project tree or use the network unless the operator explicitly permits subprocess networking. +Subagent local tools and subprocesses remain kernel-confined read-only and +networkless regardless of that permission; any web access stays brokered. fugu is neither a daemon nor a service. fugu is a clean-room reimplementation, in behavior rather than source, of blob - a5a5f5c4e5ff7c0f0dfc5faae8ad781e132c5dda blob + 004de43f19fb128200743738afea48b0a2631923 --- regress/Makefile +++ regress/Makefile @@ -1,4 +1,4 @@ -SUBDIR= anthropic buf conf deploy diff editor http journal json log markdown \ - openai output print sandbox skills sse term tools turn web xmalloc +SUBDIR= agentcfg anthropic buf conf deploy diff editor http journal json log markdown \ + model_window openai output print sandbox skills sse term tools turn web xmalloc .include blob - 6a70af64e6c40fa349cd985792bbde32f2447d2c blob + be40ddf2b0388e1e6dfa6ef65deb57d30eabe0da --- regress/README +++ regress/README @@ -1,7 +1,7 @@ Traceability: invariants and behaviors to the suites that prove them -(verification.md section 7). "pending" marks an obligation whose -subsystem does not exist yet; make check output should keep missing -proofs conspicuous. +(verification.md section 7). "pending" marks an obligation without its +required direct proof; make check output should keep missing proofs +conspicuous. Invariant Suite(s) --------- -------- @@ -9,9 +9,11 @@ I1 single-custodian credentials sandbox (api/web cann or open files after CA load); turn (only fugu-api attaches the key to the request) -I2 credentials at rest conf (redaction, freezero, - key-file secrecy); wipe-on- - delegation pending (M3) +I2 credentials at rest conf (redaction, freezero, key-file + secrecy; idempotent post-delegation + wipe covers flat, legacy, web, and + every named-provider key while + nonsecret routing survives) I3 config secrecy fail-closed conf (fd-based check, modes, symlink shape, key files, includes); setgid FUGU_CONF @@ -73,7 +75,27 @@ I14 resume fidelity journal (transactional replay: an empty reply is abandoned, not committed as an unusable message); print (SIGINT rollback and clean resume) -I15 subagents read-only pending (M9) +I15 subagents read-only sandbox (fugu-tool-r cannot create: + pledge/kdump proves the wpath + ceiling); conf (implicit/named + provider selection); agentcfg (bounded schema, + exact route palette and fallback); + turn (real ephemeral worker, read- + only tool surface and local loop, + no nesting, credential brokering, + progress, lead-only journal boundary, + hostile label and invalid route make + no child request, + cap-1 waves, cap-2 correlated provider + overlap/reverse completion, serialized + child web calls, saturated-pool backoff + release, provider and 25-round conclusions, + oversized conclusion/error tail retention, + aggregate input/ + result bounds, SIGINT child-group reap, + >16-KiB dynamic protect snapshot, escaped + future literals, and + network/shell-write denial) I16 invariants demonstrable sandbox (ktrace/kdump PLDG records); grows with each kernel-enforced wall @@ -89,12 +111,14 @@ provider codec: anthropic anthropic (builder + stream prompt-caching breakpoints) provider codec: openai openai (builder, adversarial stream, Bearer-auth full worker mesh) -tools (behavior.md 5) tools (executors, caps, freshness, +tools (behavior.md 5) agentcfg (agent schema and routing); + tools (executors, caps, freshness, NUL rejection, bounded background draining, supervisor/liveness cleanup); web (search/fetch/request policy, caps, credential scoping); turn - (a full file-tool loop over TLS) + (full file and ephemeral-agent loops + over concurrent TLS connections) headless turn (behavior.md 1) turn, print (real -p turns over TLS: plain/machine output, all pinned exit codes, key flow, transactional cancel) @@ -103,17 +127,22 @@ line mode (behavior.md 2.2) turn (multi-turn piped se slash commands (behavior.md 3) turn, print (/clear, /context, /compact, /quit and Skill dispatch); journal (clear replay) +effective context window (behavior.md 3) model_window (metadata parsing and + precedence); turn (selected picker + metadata and context_limit override) skills (behavior.md 4) skills (loader/parser edges); turn (/name dispatch, skill tool) project context (behavior.md 3/4) turn (FUGU.md injection, # capture, project_context no); journal (context/compact replay) piped stdin context (behavior.md 1) turn (fenced block on the wire) -provider error surfacing (behavior.md 8) turn (401 body reaches stderr - with the provider's type/message) -transient retry policy (behavior.md 8) turn (503 retried with notices, +provider error surfacing (behavior.md 8) turn, print (401 body reaches stderr + with the provider's type/message; long-key + SSE/HTTP reflection fails closed) +transient retry policy (behavior.md 8) turn, print (503 retried with notices, exhaustion after 5, Retry-After - honored and clamped) + honored and clamped; truncated and + slow-drip error bodies stay retryable) session journal (behavior.md 7) journal, turn session listing (behavior.md 1) journal, turn (plain, JSON, NDJSON) machine output (behavior.md 1) output (schema/hostile bytes); print @@ -126,22 +155,27 @@ src/common/xmalloc.c xmalloc src/common/log.c log src/common/buf.c buf src/common/json.c (+ jsmn) json +src/common/model_window.c model_window; turn (picker propagation) src/common/sse.c sse -src/common/http.c http (hermetic TLS stub) +src/common/http.c http (hermetic TLS stub, strict chunk + data delimiters and bounded trailers) src/common/html.c, web_url.c web (hostile reducer, URL/ACL/SSRF) src/common/msg.c, anthropic.c anthropic src/common/openai_req.c, openai_stream.c openai src/fugu/parse.y, conf.c conf +src/fugu/agentcfg.c agentcfg; turn (live schema/route) src/fugu/priv.c, coord.c sandbox (spawn/handshake/teardown), - turn (turn loop) + turn (lead and subagent turn loops) src/fugu/journal.c journal, turn src/common/imsgev.c, worker.c sandbox (worker runtime) -src/fugu-api (request/stream) turn, openai (real requests + SSE) +src/fugu-api (request/stream) turn (correlated concurrent lead/agent + requests), print (deadline/retry/long-key + reflection), openai (real requests + SSE) src/fugu-tool (tools.c, job.c, - supervisor.c) tools (executors + jobs); sandbox + supervisor.c, agent.c) tools (executors + jobs); sandbox (confinement); turn (tool loop, - CTX read/append for context) + CTX read/append, ephemeral agent loop) src/fugu/tooldefs.c turn (tools offered per gates, the skill tool) src/fugu/skills.c skills (load/parse); turn (dispatch) blob - /dev/null blob + 6b11c5eaf107a35b203295e456c5213e0d309802 (mode 644) --- /dev/null +++ regress/agentcfg/Makefile @@ -0,0 +1,13 @@ +PROG= agentcfg_test +SRCS= agentcfg_test.c agentcfg.c conf.c json.c buf.c xmalloc.c log.c +NOMAN= yes + +CFLAGS+= -I${.CURDIR}/../../src/fugu +.PATH: ${.CURDIR}/../../src/fugu + +REGRESS_TARGETS= run-agentcfg + +run-agentcfg: agentcfg_test + ./agentcfg_test + +.include blob - /dev/null blob + e832b8c02e65a67a9429a48f3d6f294dcae47f5b (mode 644) --- /dev/null +++ regress/agentcfg/agentcfg_test.c @@ -0,0 +1,357 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include +#include + +#include "msg.h" +#include "conf.h" +#include "agentcfg.h" +#include "json.h" +#include "log.h" +#include "regress.h" + +static int +has_choice(const struct agentcfg *ac, const char *want) +{ + size_t i; + + for (i = 0; i < agentcfg_nchoices(ac); i++) + if (strcmp(agentcfg_choice(ac, i), want) == 0) + return (1); + return (0); +} + +static void +init_conf(struct fugu_conf *cf, const char *model) +{ + memset(cf, 0, sizeof(*cf)); + TAILQ_INIT(&cf->providers); + cf->model = (char *)model; + cf->provider_type = PROVIDER_ANTHROPIC; + cf->api_key = "test-key"; + cf->max_subagents = 4; +} + +static void +add_provider(struct fugu_conf *cf, struct fugu_provider *p, + const char *name, int type, const char *model) +{ + memset(p, 0, sizeof(*p)); + p->name = (char *)name; + p->type = type; + p->model = (char *)model; + TAILQ_INSERT_TAIL(&cf->providers, p, entry); +} + +static void +test_schema_and_palette(void) +{ + struct fugu_provider p1, p2; + struct fugu_conf cf; + struct agentcfg *ac; + struct agentcfg_route route; + const struct tool_def *tool; + const char hostile[] = "quo\"te\\line\nctl\001:m"; + + init_conf(&cf, hostile); + ac = agentcfg_new(&cf); + CHECK(ac != NULL); + tool = agentcfg_tool(ac, 0, PROVIDER_ANTHROPIC, "live"); + CHECK(tool != NULL && strcmp(tool->name, "agent") == 0); + CHECK(tool != NULL && strstr(tool->schema, + "\"required\":[\"prompt\"]") != NULL); + CHECK(tool != NULL && json_valid_object(tool->schema, + strlen(tool->schema))); + CHECK(tool != NULL && strstr(tool->schema, + "\"label\":{\"type\":\"string\"") != NULL); + CHECK(tool != NULL && strstr(tool->schema, + "quo\\\"te\\\\line\\nctl\\u0001:m") != NULL); + CHECK(tool != NULL && strstr(tool->description, "parallel") != NULL && + strstr(tool->description, "read-only") != NULL && + strstr(tool->description, "self-contained") != NULL && + strstr(tool->description, "final report") != NULL && + strstr(tool->description, "wait for a slot") != NULL && + strstr(tool->description, "lighter model") != NULL && + strstr(tool->description, "stronger one") != NULL); + CHECK(agentcfg_nchoices(ac) == 1); + CHECK(strcmp(agentcfg_choice(ac, 0), hostile) == 0); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "live", + hostile, &route) == 0 && route.provider_slot == 0 && + route.provider_type == PROVIDER_ANTHROPIC && + strcmp(route.model, hostile) == 0); + agentcfg_free(ac); + + /* Invalid UTF-8 is normalized by the one project-wide JSON codec. */ + init_conf(&cf, "bad\xff:model"); + ac = agentcfg_new(&cf); + tool = agentcfg_tool(ac, 0, PROVIDER_ANTHROPIC, "live"); + CHECK(tool != NULL && json_valid_object(tool->schema, + strlen(tool->schema))); + CHECK(agentcfg_nchoices(ac) == 1 && strcmp(agentcfg_choice(ac, 0), + "bad\xef\xbf\xbd:model") == 0); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "live", + "bad\xef\xbf\xbd:model", &route) == 0 && + strcmp(route.model, "bad\xef\xbf\xbd:model") == 0); + agentcfg_free(ac); + + /* Multiple slots qualify every entry; ':' in a model is opaque. */ + init_conf(&cf, "claude:base"); + add_provider(&cf, &p1, "work", PROVIDER_OPENAI, "gpt:5/x"); + add_provider(&cf, &p2, "default", PROVIDER_ANTHROPIC, "same"); + cf.subagent_model = "mini:one"; + ac = agentcfg_new(&cf); + CHECK(ac != NULL); + + /* A later /model mutation cannot replace the configured snapshot. */ + cf.model = "mutated-live-model"; + tool = agentcfg_tool(ac, 1, PROVIDER_OPENAI, cf.model); + CHECK(tool != NULL && agentcfg_nchoices(ac) == 4); + CHECK(has_choice(ac, "default:claude:base")); + CHECK(has_choice(ac, "work:gpt:5/x")); + CHECK(has_choice(ac, "@default:same")); + CHECK(has_choice(ac, "work:mini:one")); + CHECK(!has_choice(ac, "default:mutated-live-model")); + CHECK(agentcfg_resolve(ac, 1, PROVIDER_OPENAI, cf.model, + "work:gpt:5/x", &route) == 0 && route.provider_slot == 1 && + route.provider_type == PROVIDER_OPENAI && + strcmp(route.model, "gpt:5/x") == 0); + CHECK(agentcfg_resolve(ac, 1, PROVIDER_OPENAI, cf.model, + "work:gpt", &route) == -1); + CHECK(agentcfg_resolve(ac, 1, PROVIDER_OPENAI, cf.model, + "@default:same", &route) == 0 && route.provider_slot == 2 && + strcmp(route.provider_name, "default") == 0 && + strcmp(route.model, "same") == 0); + + /* subagent_model without a provider follows a new live lead route. */ + tool = agentcfg_tool(ac, 2, PROVIDER_ANTHROPIC, "other-live"); + CHECK(tool != NULL && has_choice(ac, "@default:mini:one")); + CHECK(!has_choice(ac, "work:mini:one")); + CHECK(agentcfg_resolve(ac, 2, PROVIDER_ANTHROPIC, "other-live", + NULL, &route) == 0 && route.provider_slot == 2 && + strcmp(route.model, "mini:one") == 0); + agentcfg_free(ac); + + /* A parser-selected named provider means credentialless slot zero is + * neither advertised nor accepted, even if its live fields were replaced + * with the named provider's dialect/model during startup selection. */ + init_conf(&cf, "work-model"); + cf.api_key = NULL; + cf.initial_provider_slot = 1; + cf.provider_type = PROVIDER_OPENAI; + add_provider(&cf, &p1, "work", PROVIDER_OPENAI, "work-model"); + ac = agentcfg_new(&cf); + CHECK(ac != NULL); + CHECK(agentcfg_tool(ac, 1, PROVIDER_OPENAI, "work-model") != NULL); + CHECK(agentcfg_nchoices(ac) == 1); + CHECK(has_choice(ac, "work:work-model")); + CHECK(!has_choice(ac, "default:work-model")); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_OPENAI, "work-model", NULL, + &route) == -1); + agentcfg_free(ac); +} + +static void +test_default_matrix(void) +{ + struct fugu_provider p; + struct fugu_conf cf; + struct agentcfg *ac; + struct agentcfg_route route; + + /* Neither override: both components follow the lead independently. */ + init_conf(&cf, "configured"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, "work-model"); + ac = agentcfg_new(&cf); + CHECK(agentcfg_resolve(ac, 1, PROVIDER_OPENAI, "live", NULL, + &route) == 0 && route.provider_slot == 1 && + route.provider_type == PROVIDER_OPENAI && + strcmp(route.model, "live") == 0); + agentcfg_free(ac); + + /* Model only: live provider plus fixed model. */ + init_conf(&cf, "configured"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, "work-model"); + cf.subagent_model = "small"; + ac = agentcfg_new(&cf); + CHECK(agentcfg_resolve(ac, 1, PROVIDER_OPENAI, "live", NULL, + &route) == 0 && route.provider_slot == 1 && + route.provider_type == PROVIDER_OPENAI && + strcmp(route.model, "small") == 0); + agentcfg_free(ac); + + /* Provider only: fixed provider plus live model. */ + init_conf(&cf, "configured"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, "work-model"); + cf.subagent_provider = "work"; + ac = agentcfg_new(&cf); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "live", NULL, + &route) == 0 && route.provider_slot == 1 && + route.provider_type == PROVIDER_OPENAI && + strcmp(route.model, "live") == 0); + agentcfg_free(ac); + + /* Both overrides: the entire route is fixed. */ + init_conf(&cf, "configured"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, "work-model"); + cf.subagent_provider = "work"; + cf.subagent_model = "small"; + ac = agentcfg_new(&cf); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "live", NULL, + &route) == 0 && route.provider_slot == 1 && + route.provider_type == PROVIDER_OPENAI && + strcmp(route.model, "small") == 0); + agentcfg_free(ac); + + /* @default is the explicit, unambiguous spelling for slot zero. */ + init_conf(&cf, "configured"); + add_provider(&cf, &p, "default", PROVIDER_OPENAI, "named-model"); + cf.subagent_provider = FUGU_IMPLICIT_PROVIDER_REF; + cf.subagent_model = "small"; + ac = agentcfg_new(&cf); + CHECK(agentcfg_resolve(ac, 1, PROVIDER_OPENAI, "live", NULL, + &route) == 0 && route.provider_slot == 0 && + route.provider_type == PROVIDER_ANTHROPIC && + strcmp(route.model, "small") == 0); + agentcfg_free(ac); + + /* The unadorned name still addresses a block literally named default. */ + init_conf(&cf, "configured"); + add_provider(&cf, &p, "default", PROVIDER_OPENAI, "named-model"); + cf.subagent_provider = "default"; + ac = agentcfg_new(&cf); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "live", NULL, + &route) == 0 && route.provider_slot == 1 && + route.provider_type == PROVIDER_OPENAI && + strcmp(route.provider_name, "default") == 0 && + strcmp(route.model, "live") == 0); + agentcfg_free(ac); +} + +static void +test_gate_dedup_and_invalid(void) +{ + struct fugu_provider p; + struct fugu_conf cf; + struct agentcfg *ac; + struct agentcfg_route route; + const char *borrowed; + + init_conf(&cf, "same"); + cf.subagent_model = "same"; + ac = agentcfg_new(&cf); + CHECK(agentcfg_tool(ac, 0, PROVIDER_ANTHROPIC, "same") != NULL); + CHECK(agentcfg_nchoices(ac) == 1); + borrowed = agentcfg_choice(ac, 0); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "same", borrowed, + &route) == 0 && strcmp(route.model, "same") == 0); + CHECK(agentcfg_resolve(ac, 9, PROVIDER_OPENAI, "live", NULL, + &route) == -1); + agentcfg_free(ac); + + /* Equal IDs on different routes are deliberately distinct choices. */ + init_conf(&cf, "same"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, "same"); + ac = agentcfg_new(&cf); + CHECK(agentcfg_tool(ac, 0, PROVIDER_ANTHROPIC, "same") != NULL && + agentcfg_nchoices(ac) == 2); + CHECK(has_choice(ac, "default:same") && has_choice(ac, "work:same")); + agentcfg_free(ac); + + init_conf(&cf, "same"); + cf.max_subagents = 0; + ac = agentcfg_new(&cf); + CHECK(ac != NULL && agentcfg_tool(ac, 0, PROVIDER_ANTHROPIC, + "same") == NULL); + CHECK(agentcfg_resolve(ac, 0, PROVIDER_ANTHROPIC, "same", NULL, + &route) == -1); + agentcfg_free(ac); + + /* Only parsed provider names may be referenced. */ + init_conf(&cf, "same"); + cf.subagent_provider = "missing"; + CHECK(agentcfg_new(&cf) == NULL); + + /* The implicit reference also fails closed when slot zero has no key. */ + init_conf(&cf, "same"); + cf.api_key = NULL; + cf.initial_provider_slot = 1; + add_provider(&cf, &p, "work", PROVIDER_OPENAI, "same"); + cf.subagent_provider = FUGU_IMPLICIT_PROVIDER_REF; + CHECK(agentcfg_new(&cf) == NULL); +} + +static void +test_model_bounds(void) +{ + struct fugu_provider p; + struct fugu_conf cf; + struct agentcfg *ac; + char model[FUGU_MODEL_ID_MAX + 2]; + char expanded[(FUGU_MODEL_ID_MAX / 3) + 2]; + + memset(model, 'm', FUGU_MODEL_ID_MAX); + model[FUGU_MODEL_ID_MAX] = '\0'; + init_conf(&cf, model); + ac = agentcfg_new(&cf); + CHECK(ac != NULL); + CHECK(agentcfg_tool(ac, 0, PROVIDER_ANTHROPIC, model) != NULL); + CHECK(agentcfg_nchoices(ac) == 1 && + strlen(agentcfg_choice(ac, 0)) == FUGU_MODEL_ID_MAX); + agentcfg_free(ac); + + model[FUGU_MODEL_ID_MAX] = 'x'; + model[FUGU_MODEL_ID_MAX + 1] = '\0'; + init_conf(&cf, model); + CHECK(agentcfg_new(&cf) == NULL); + + init_conf(&cf, "valid"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, model); + CHECK(agentcfg_new(&cf) == NULL); + + init_conf(&cf, "valid"); + cf.subagent_model = model; + CHECK(agentcfg_new(&cf) == NULL); + + init_conf(&cf, ""); + CHECK(agentcfg_new(&cf) == NULL); + init_conf(&cf, "valid"); + add_provider(&cf, &p, "work", PROVIDER_OPENAI, ""); + CHECK(agentcfg_new(&cf) == NULL); + init_conf(&cf, "valid"); + cf.subagent_model = ""; + CHECK(agentcfg_new(&cf) == NULL); + + /* JSON normalization may expand invalid UTF-8 beyond the wire bound. */ + memset(expanded, 0xff, sizeof(expanded) - 1); + expanded[sizeof(expanded) - 1] = '\0'; + init_conf(&cf, expanded); + CHECK(agentcfg_new(&cf) == NULL); +} + +int +main(void) +{ + log_init(LOG_TO_STDERR, 0, LOG_DAEMON); + test_schema_and_palette(); + test_default_matrix(); + test_gate_dedup_and_invalid(); + test_model_bounds(); + REGRESS_END(); +} blob - 54c8854efc785197b214d659afbe94b0cd40a345 blob + e82717f33f1bdad1b6bbbd22132898046127b238 --- regress/buf/buf_test.c +++ regress/buf/buf_test.c @@ -72,6 +72,16 @@ main(void) log_init(LOG_TO_STDERR, 0, LOG_USER); + /* Callers can reserve aggregate memory against the allocator's exact + * growth rule before accepting hostile bytes. */ + buf_init(&b); + CHECK(buf_capacity_after(&b, 0) == 0); + CHECK(buf_capacity_after(&b, 1) == 64); + buf_addc(&b, 'x'); + CHECK(buf_capacity_after(&b, 63) == 64); + CHECK(buf_capacity_after(&b, 64) == 128); + buf_free(&b); + /* growth across many single-byte appends */ buf_init(&b); for (i = 0; i < 10000; i++) blob - 2494ab8478f0d4db69574ea6680dacfceb856f7b blob + 613d2972a0135cf324f427183c4c432c5fd82857 --- regress/conf/conf_test.c +++ regress/conf/conf_test.c @@ -461,9 +461,31 @@ test_providers(void) CHECK(parse_fails("provider \"p\" {\n type \"weird\"\n" " api_key \"k\"\n}\n")); - /* subagent_provider must name a configured block */ + /* subagent_provider names a block, or @default for usable slot zero. */ CHECK(parse_fails("subagent_provider \"nosuch\"\n")); + CHECK(parse_fails( + "provider \"w\" {\n\tapi_key \"k\"\n}\n" + "subagent_provider \"@default\"\n")); CHECK(parse_ok( + "api_key \"flat\"\n" + "provider \"w\" {\n\tapi_key \"k\"\n}\n" + "subagent_provider \"@default\"\n", &alice, &c)); + CHECK(strcmp(c->subagent_provider, FUGU_IMPLICIT_PROVIDER_REF) == 0); + conf_free(c); + /* A block literally named default remains addressable without ambiguity. */ + CHECK(parse_ok( + "api_key \"flat\"\n" + "provider \"default\" {\n\tapi_key \"named\"\n}\n" + "subagent_provider \"default\"\n", &alice, &c)); + CHECK(strcmp(c->subagent_provider, "default") == 0); + conf_free(c); + /* A legacy Anthropic key cannot key an implicit OpenAI provider. */ + CHECK(parse_fails( + "provider \"openai\"\n" + "anthropic_key \"legacy\"\n" + "provider \"w\" {\n\tapi_key \"k\"\n}\n" + "subagent_provider \"@default\"\n")); + CHECK(parse_ok( "provider \"w\" {\n api_key \"k\"\n}\n" "subagent_provider \"w\"\n" "subagent_model \"claude-haiku-4-5\"\n", &alice, &c)); @@ -500,6 +522,45 @@ test_numbers(void) } static void +test_model_bounds(void) +{ + struct fugu_conf *c; + struct fugu_provider *p; + char cfg[1024]; + char model[FUGU_MODEL_ID_MAX + 2]; + + memset(model, 'm', FUGU_MODEL_ID_MAX); + model[FUGU_MODEL_ID_MAX] = '\0'; + snprintf(cfg, sizeof(cfg), + "model \"%s\"\n" + "api_key \"flat\"\n" + "provider \"work\" {\n model \"%s\"\n api_key \"named\"\n}\n" + "subagent_model \"%s\"\n", model, model, model); + CHECK(parse_ok(cfg, &alice, &c)); + p = TAILQ_FIRST(&c->providers); + CHECK(strlen(c->model) == FUGU_MODEL_ID_MAX); + CHECK(p != NULL && strlen(p->model) == FUGU_MODEL_ID_MAX); + CHECK(strlen(c->subagent_model) == FUGU_MODEL_ID_MAX); + conf_free(c); + + model[FUGU_MODEL_ID_MAX] = 'x'; + model[FUGU_MODEL_ID_MAX + 1] = '\0'; + snprintf(cfg, sizeof(cfg), "model \"%s\"\n", model); + CHECK(parse_fails(cfg)); + snprintf(cfg, sizeof(cfg), + "provider \"work\" {\n model \"%s\"\n api_key \"k\"\n}\n", + model); + CHECK(parse_fails(cfg)); + snprintf(cfg, sizeof(cfg), "subagent_model \"%s\"\n", model); + CHECK(parse_fails(cfg)); + + CHECK(parse_fails("model \"\"\n")); + CHECK(parse_fails( + "provider \"work\" {\n model \"\"\n api_key \"k\"\n}\n")); + CHECK(parse_fails("subagent_model \"\"\n")); +} + +static void test_hostile(void) { char big[16384]; @@ -615,6 +676,34 @@ test_dump_redacts(void) } static void +test_wipe_delegated_secrets(void) +{ + struct fugu_conf *c; + struct fugu_provider *p; + + CHECK(parse_ok( + "model \"keep-model\"\n" + "api_key \"flat-secret\"\n" + "anthropic_key \"legacy-secret\"\n" + "kagi_token \"web-secret\"\n" + "provider \"work\" {\n" + "\tapi_key \"named-secret\"\n" + "}\n", &alice, &c)); + p = TAILQ_FIRST(&c->providers); + CHECK(c->api_key != NULL && c->anthropic_key != NULL && + c->kagi_token != NULL && p != NULL && p->api_key != NULL); + conf_wipe_secrets(c); + CHECK(c->api_key == NULL && c->anthropic_key == NULL && + c->kagi_token == NULL && p->api_key == NULL); + CHECK(strcmp(c->model, "keep-model") == 0 && + strcmp(p->name, "work") == 0); + /* Idempotence matters on startup-error and teardown paths. */ + conf_wipe_secrets(c); + CHECK(c->api_key == NULL && p->api_key == NULL); + conf_free(c); +} + +static void test_lifetimes(void) { struct fugu_conf *c; @@ -659,9 +748,11 @@ main(void) test_providers(); test_macros(); test_numbers(); + test_model_bounds(); test_hostile(); test_include(); test_dump_redacts(); + test_wipe_delegated_secrets(); test_lifetimes(); { blob - 115e8fa4f976a9c4841f5aae0c4e68deea059f39 blob + c7b67bafaa5a1056c0f212c5a6810362cc3265bb --- regress/http/http_test.c +++ regress/http/http_test.c @@ -98,11 +98,23 @@ read_all(struct http *h, struct buf *out) } } +static void +init_chunked(struct http *h) +{ + static const char hdr[] = + "HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n"; + + http_init(h); + buf_add(&h->in, hdr, sizeof(hdr) - 1); + CHECK(http_response_parse(h, 0) == 1); + CHECK(h->chunked); +} + int main(void) { struct http h; - struct buf body; + struct buf body, head; const char *val; size_t vlen; @@ -121,6 +133,97 @@ main(void) badcfg = make_config(badca); buf_init(&body); + /* Async callers build and wipe only the header, then stream the retained + * body without allocating a second full-size copy. */ + http_init(&h); + h.host = xstrdup("example.test"); + h.port = xstrdup("443"); + buf_init(&head); + CHECK(http_request_head_build(&h, &head, "POST", 4, "/v1", 3, + "x-test: yes\r\n", 13, 3) == 0); + CHECK(memmem(head.data, head.len, "Content-Length: 3\r\n", 19) != NULL); + CHECK(head.len >= 4 && memcmp(head.data + head.len - 4, + "\r\n\r\n", 4) == 0); + CHECK(memmem(head.data, head.len, "abc", 3) == NULL); + buf_free(&head); + http_close(&h); + + /* Incremental chunk framing: the data delimiter may be split across + * reads, and trailers are consumed before end-of-body is reported. */ + { + char c; + + init_chunked(&h); + buf_addstr(&h.in, "1\r\na"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == 1 && c == 'a'); + buf_addstr(&h.in, "\r"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == HTTP_BODY_MORE); + buf_addstr(&h.in, "\n0\r\nX-Trace: yes\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == HTTP_BODY_MORE); + buf_addstr(&h.in, "\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == 0); + CHECK(h.body_eof && h.in.len == 0); + http_close(&h); + } + + /* A following chunk-size line cannot stand in for the required CRLF. */ + { + char c; + + init_chunked(&h); + buf_addstr(&h.in, "1\r\na0\r\n\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == 1 && c == 'a'); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == -1); + CHECK(strstr(http_error(&h), "CRLF") != NULL); + http_close(&h); + } + + /* Blank size lines and malformed/forbidden trailer fields fail closed. */ + { + char c; + + init_chunked(&h); + buf_addstr(&h.in, "\r\n0\r\n\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == -1); + CHECK(strstr(http_error(&h), "chunk size") != NULL); + http_close(&h); + + init_chunked(&h); + buf_addstr(&h.in, "0\r\nnot-a-field\r\n\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == -1); + CHECK(strstr(http_error(&h), "trailer") != NULL); + http_close(&h); + + init_chunked(&h); + buf_addstr(&h.in, "0\r\nContent-Length: 1\r\n\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == -1); + CHECK(strstr(http_error(&h), "forbidden") != NULL); + http_close(&h); + } + + /* The final blank line is mandatory and the whole trailer section is + * bounded even when a peer never supplies another newline. */ + { + char c; + char *large; + + init_chunked(&h); + buf_addstr(&h.in, "0\r\nX-Trace: yes\r\n"); + CHECK(http_body_parse(&h, &c, sizeof(c), 1) == -1); + CHECK(strstr(http_error(&h), "truncated") != NULL); + http_close(&h); + + init_chunked(&h); + buf_addstr(&h.in, "0\r\n"); + large = xmalloc(HTTP_HDR_MAX + 1); + memset(large, 'a', HTTP_HDR_MAX + 1); + buf_add(&h.in, large, HTTP_HDR_MAX + 1); + free(large); + CHECK(http_body_parse(&h, &c, sizeof(c), 0) == -1); + CHECK(strstr(http_error(&h), "trailers too large") != NULL); + http_close(&h); + } + /* r1: content-length body */ CHECK(open_get(&h, "/r1") == 0); CHECK(h.status == 200); blob - /dev/null blob + 6b586c89c2fbc15f8cce8e4d6ae2ef7ad68cabb1 (mode 644) --- /dev/null +++ regress/model_window/Makefile @@ -0,0 +1,10 @@ +PROG= model_window_test +SRCS= model_window_test.c model_window.c json.c buf.c log.c xmalloc.c +NOMAN= yes + +REGRESS_TARGETS= run-model-window + +run-model-window: model_window_test + ./model_window_test + +.include blob - /dev/null blob + 6b3aca0ae076aa623a553cedcb7cab171b9baa2d (mode 644) --- /dev/null +++ regress/model_window/model_window_test.c @@ -0,0 +1,80 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include + +#include "json.h" +#include "model_window.h" +#include "regress.h" + +static int64_t +parse_window(const char *s) +{ + struct json j; + int64_t v; + + if (json_parse(&j, s, strlen(s), 0) != 0) + return (-1); + v = model_window_parse(&j, json_root(&j)); + json_done(&j); + return (v); +} + +static void +test_provider_metadata(void) +{ + CHECK(parse_window("{\"context_window\":200000}") == 200000); + CHECK(parse_window("{\"context_length\":128000}") == 128000); + CHECK(parse_window("{\"max_input_tokens\":64000}") == 64000); + CHECK(parse_window("{\"context_window\":111," + "\"context_length\":222,\"max_input_tokens\":333}") == 111); + CHECK(parse_window("{\"context_window\":1000000001," + "\"context_length\":424242}") == 424242); + CHECK(parse_window("{\"context_window\":\"200000\"," + "\"max_input_tokens\":32000}") == 32000); + CHECK(parse_window("{\"context_window\":1.5," + "\"context_length\":0,\"max_input_tokens\":-1}") == 0); + CHECK(parse_window("{\"context_window\":" + "999999999999999999999999999999999999}") == 0); + CHECK(parse_window("[]") == 0); +} + +static void +test_effective_order(void) +{ + /* Operator override wins over selected metadata and the table. */ + CHECK(model_window_effective(777777, 424242, + "claude-known") == 777777); + /* The selected provider's value wins over a compiled known model. */ + CHECK(model_window_effective(0, 424242, + "claude-known") == 424242); + CHECK(model_window_effective(0, 0, "claude-known") == 200000); + CHECK(model_window_effective(0, 0, "gpt-known") == 128000); + CHECK(model_window_effective(0, 0, "unlisted-model") == 0); + CHECK(model_window_effective(0, FUGU_MODEL_WINDOW_MAX + 1, + "gpt-known") == 128000); +} + +int +main(void) +{ + test_provider_metadata(); + test_effective_order(); + REGRESS_END(); +} blob - ce2e2d41ac35999fc556238cbe8f825a38b19aa1 blob + 449dfa376277c752cbcbc4975e4e561268c783fd --- regress/print/run.sh +++ regress/print/run.sh @@ -168,6 +168,85 @@ ok "secret: error reflection retains machine error sha ok "secret: reflected error key never crosses into output or journal" $? stop_stub +# A provider reflecting a long credential must not exploit error truncation to +# leak a prefix, whether the error arrives in SSE or a non-2xx JSON body. +longprefix='LONG-CREDENTIAL-PREFIX-MUST-NOT-LEAK-' +longkey=$longprefix +while [ "${#longkey}" -le 1536 ]; do + longkey="${longkey}xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" +done + +reset_home +start_stub "REFLECT_LONG_SSE" +write_conf "$longkey" +run_fugu -p "reflect a long SSE credential" -o ndjson +ok "secret: long SSE reflection fails the Turn" \ + $([ "$rc" -eq 1 ]; echo $?) +"$obj/jsoncheck" error "$dir/out" +ok "secret: long SSE reflection retains machine error shape" $? +grep -q "$longprefix" "$dir/reqout" 2>/dev/null +ok "secret: long key reached the credential-holding provider" $? +! grep -q "$longprefix" "$dir/out" "$dir/errout" \ + "$sessions"/*.ndjson 2>/dev/null +ok "secret: long SSE reflection leaks no credential prefix" $? +stop_stub + +reset_home +start_stub "REFLECT_LONG_HTTP" +write_conf "$longkey" +run_fugu -p "reflect a long HTTP credential" -o ndjson +ok "secret: long HTTP reflection fails the Turn" \ + $([ "$rc" -eq 1 ]; echo $?) +"$obj/jsoncheck" error "$dir/out" +ok "secret: long HTTP reflection retains machine error shape" $? +grep -q "$longprefix" "$dir/reqout" 2>/dev/null +ok "secret: long key reached the non-2xx provider path" $? +! grep -q "$longprefix" "$dir/out" "$dir/errout" \ + "$sessions"/*.ndjson 2>/dev/null +ok "secret: long HTTP reflection leaks no credential prefix" $? +stop_stub + +# Once a retryable status is known, malformed error framing cannot downgrade +# it to a terminal transport error: the identical request is retried. +reset_home +start_stub "TRUNC503 OK" +write_conf +run_fugu -p "retry a truncated 503" -o json +ok "retry: truncated 503 succeeds on the next attempt" \ + $([ "$rc" -eq 0 ]; echo $?) +"$obj/jsoncheck" shape "$dir/out" +ok "retry: truncated 503 preserves final JSON shape" $? +grep -q '^2$' "$dir/reqcount" 2>/dev/null && \ + grep -q 'retry 2/5' "$dir/errout" 2>/dev/null +ok "retry: truncated 503 emitted one notice and a second request" $? +stop_stub + +# Five bytes arriving each second defeat an inactivity-only timer. Completion +# near five seconds demonstrates the independent aggregate error-body cap. +reset_home +start_stub "DRIP503 OK" +write_conf +started=$(date +%s) +run_fugu -p "retry a slow-drip 503" -o json +finished=$(date +%s) +elapsed=$((finished - started)) +ok "retry: slow-drip 503 succeeds after its aggregate deadline" \ + $([ "$rc" -eq 0 ]; echo $?) +"$obj/jsoncheck" shape "$dir/out" +ok "retry: slow-drip 503 preserves final JSON shape" $? +grep -q '^2$' "$dir/reqcount" 2>/dev/null && \ + grep -q 'retry 2/5' "$dir/errout" 2>/dev/null +ok "retry: slow-drip deadline caused a second request" $? +[ "$elapsed" -ge 4 ] && [ "$elapsed" -le 12 ] +if [ $? -ne 0 ]; then + echo "# slow-drip elapsed: ${elapsed}s" + timing_rc=1 +else + timing_rc=0 +fi +ok "retry: slow-drip error body is bounded near five seconds" "$timing_rc" +stop_stub + # Malformed provider tool JSON is rejected before execution/journaling, and # an abandoned attempt cannot create an orphaned result on resume (I14). reset_home blob - e9c3639993671fe2a93c05f334030e74e7ef1627 blob + 1ef69866f1786c67f6d3d34dfa58491d9b6a69d5 --- regress/print/stub.c +++ regress/print/stub.c @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -29,6 +30,10 @@ #define REQUEST_MAX (2 * 1024 * 1024) +static ssize_t read_request(struct tls *, char *, size_t); +static void record_request(const char *, const char *, size_t); +static void record_count(const char *, int); + static const char http_ok[] = "HTTP/1.1 200 OK\r\n" "content-type: text/event-stream\r\n" @@ -163,7 +168,123 @@ send_sse(struct tls *conn, const char *sse) (void)tls_write_all(conn, "0\r\n\r\n", 5); } +static char * +request_api_key(const char *request) +{ + const char *h, *end; + char *key; + size_t len; + + if ((h = strcasestr(request, "x-api-key:")) == NULL) + return (NULL); + h += strlen("x-api-key:"); + while (*h == ' ' || *h == '\t') + h++; + if ((end = strstr(h, "\r\n")) == NULL || end == h) + return (NULL); + len = (size_t)(end - h); + key = malloc(len + 1); + if (key == NULL) + err(1, "malloc reflected key"); + memcpy(key, h, len); + key[len] = '\0'; + return (key); +} + static void +send_reflected_sse_error(struct tls *conn, const char *key) +{ + char *sse; + + if (asprintf(&sse, + "event: error\n" + "data: {\"type\":\"error\",\"error\":{" + "\"type\":\"provider_error\",\"message\":\"%s\"}}\n\n", + key) == -1) + err(1, "asprintf reflected SSE error"); + send_sse(conn, sse); + free(sse); +} + +static void +send_reflected_http_error(struct tls *conn, const char *key) +{ + char *body; + char hdr[256]; + int n; + + if (asprintf(&body, + "{\"type\":\"error\",\"error\":{\"type\":\"auth_error\"," + "\"message\":\"%s\"}}", key) == -1) + err(1, "asprintf reflected HTTP error"); + n = snprintf(hdr, sizeof(hdr), + "HTTP/1.1 401 Unauthorized\r\n" + "content-type: application/json\r\n" + "content-length: %zu\r\n\r\n", strlen(body)); + if (n < 0 || (size_t)n >= sizeof(hdr)) + errx(1, "reflected HTTP header too large"); + (void)tls_write_all(conn, hdr, (size_t)n); + (void)tls_write_all(conn, body, strlen(body)); + free(body); +} + +static void +send_truncated_503(struct tls *conn) +{ + static const char response[] = + "HTTP/1.1 503 Service Unavailable\r\n" + "content-type: application/json\r\n" + "content-length: 100\r\n\r\n{"; + + (void)tls_write_all(conn, response, sizeof(response) - 1); +} + +/* Five body bytes per second stay inside the per-read timeout. The client + * must nevertheless close this attempt at its five-second aggregate cap. */ +static int +send_drip_503(struct tls *conn, int listener, struct tls *server, + char *request, const char *reqout, const char *countfile, int count) +{ + static const char headers[] = + "HTTP/1.1 503 Service Unavailable\r\n" + "content-type: application/json\r\n" + "content-length: 100\r\n\r\n"; + struct pollfd pfd; + struct tls *retry; + ssize_t reqlen; + int c, i; + + if (tls_write_all(conn, headers, sizeof(headers) - 1) == -1) + return (0); + memset(&pfd, 0, sizeof(pfd)); + pfd.fd = listener; + pfd.events = POLLIN; + for (i = 0; i < 100; i++) { + if (tls_write_all(conn, "x", 1) == -1) + return (0); + if (poll(&pfd, 1, 200) <= 0) + continue; + if ((c = accept(listener, NULL, NULL)) == -1) + err(1, "accept slow-drip retry"); + retry = NULL; + if (tls_accept_socket(server, &retry, c) == -1) + errx(1, "TLS accept slow-drip retry"); + memset(request, 0, REQUEST_MAX); + reqlen = read_request(retry, request, REQUEST_MAX); + if (reqlen <= 0) + errx(1, "read slow-drip retry"); + record_request(reqout, request, (size_t)reqlen); + record_count(countfile, count); + send_sse(retry, sse_ok); + (void)tls_close(retry); + tls_free(retry); + close(c); + return (1); + } + return (0); +} + +static void send_tool(struct tls *conn, int number) { char sse[4096]; @@ -352,6 +473,9 @@ main(int argc, char *argv[]) fflush(stdout); for (i = 0; i < argc; i++) { + char *reflected_key = NULL; + int consumed_next = 0; + if ((c = accept(s, NULL, NULL)) == -1) err(1, "accept"); conn = NULL; @@ -377,6 +501,25 @@ main(int argc, char *argv[]) send_sse(conn, sse_reflect_error); else if (reqlen > 0 && strcmp(argv[i], "REFLECT_SHORT") == 0) send_sse(conn, sse_reflect_short); + else if (reqlen > 0 && strcmp(argv[i], "REFLECT_LONG_SSE") == 0) { + reflected_key = request_api_key(request); + if (reflected_key == NULL) + errx(1, "long SSE reflection has no API key"); + send_reflected_sse_error(conn, reflected_key); + } else if (reqlen > 0 && strcmp(argv[i], + "REFLECT_LONG_HTTP") == 0) { + reflected_key = request_api_key(request); + if (reflected_key == NULL) + errx(1, "long HTTP reflection has no API key"); + send_reflected_http_error(conn, reflected_key); + } else if (reqlen > 0 && strcmp(argv[i], "TRUNC503") == 0) + send_truncated_503(conn); + else if (reqlen > 0 && strcmp(argv[i], "DRIP503") == 0) { + if (i + 1 >= argc || strcmp(argv[i + 1], "OK") != 0) + errx(1, "DRIP503 requires a following OK scenario"); + consumed_next = send_drip_503(conn, s, srv, request, reqout, + countfile, i + 2); + } else if (reqlen > 0 && strcmp(argv[i], "BADARGS") == 0) send_sse(conn, sse_bad_args); else if (reqlen > 0 && strcmp(argv[i], "OK") == 0) @@ -393,9 +536,15 @@ main(int argc, char *argv[]) for (;;) pause(); } + if (reflected_key != NULL) { + explicit_bzero(reflected_key, strlen(reflected_key)); + free(reflected_key); + } (void)tls_close(conn); tls_free(conn); close(c); + if (consumed_next) + i++; } free(request); close(s); blob - f7ca970e6cff47b2204b499b6d56add16caacc1b blob + 6bfe7612d0ca06dc171951787945560e50a14bbd --- regress/sandbox/harness.c +++ regress/sandbox/harness.c @@ -159,7 +159,7 @@ main(int argc, char *argv[]) expect = argv[4]; /* dead | errno | ok */ if (priv_spawn(&kid, argv[1], role, rolename, 0, 0, 0, - readonly) == -1) + readonly, -1, -1) == -1) errx(2, "spawn %s", rolename); if (imsgbuf_init(&ibuf, kid.fd) == -1) errx(2, "imsgbuf_init"); blob - 859395469bd263c9644252cdd873043dd28dbfb9 blob + 825cd8859876138e433d10148b60be87682a21c1 --- regress/tools/tools_test.c +++ regress/tools/tools_test.c @@ -148,6 +148,28 @@ make_big(const char *path, size_t target) close(fd); } +static void +make_long_line(const char *path, size_t target) +{ + char chunk[4096]; + int fd; + size_t w = 0; + + memset(chunk, 'y', sizeof(chunk)); + if ((fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0644)) == -1) + err(1, "create %s", path); + while (w < target) { + size_t want = target - w; + + if (want > sizeof(chunk)) + want = sizeof(chunk); + if (write(fd, chunk, want) != (ssize_t)want) + err(1, "write %s", path); + w += want; + } + close(fd); +} + int main(int argc, char *argv[]) { @@ -172,6 +194,7 @@ main(int argc, char *argv[]) err(1, "mkdir secret"); make_file("secret/s.txt", "topsecret\n", 10, 0644); make_big("big.txt", 1536 * 1024); /* > 1 MiB */ + make_long_line("longline.txt", 1024 * 1024 + 1); make_big("bigcat.txt", 300 * 1024); /* > 256 KiB */ /* fugu-tool's confinement: the tree rwxc, system paths rx, the @@ -221,6 +244,16 @@ main(int argc, char *argv[]) CHECK(!err_flag && outhas("window truncated") && out.len <= 1024 * 1024 + 256); + /* A line longer than the byte cap cannot be resumed with the + * line-based offset. The bounded prefix must say that the remainder + * was omitted and advance past the line instead of suggesting itself. */ + run("read", "{\"path\":\"longline.txt\",\"offset\":1,\"limit\":1}"); + CHECK(!err_flag && outhas("1 lines total")); + CHECK(outhas("next offset 2") && outhas("end-of-file") && + !outhas("offset 1")); + CHECK(outhas("remainder omitted")); + CHECK(out.len > 1024 * 1024 && out.len <= 1024 * 1024 + 256); + run("read", "{\"path\":\"a\\u0000b\"}"); /* NUL in path (I13) */ CHECK(err_flag && outhas("NUL in path")); blob - 59e7f0cb6c9d526cab404930504f731381b4c74b blob + add9be71c01d35d58d42460a4dc06bc27aa4fb9c --- regress/turn/Makefile +++ regress/turn/Makefile @@ -5,7 +5,7 @@ NOMAN= yes LDADD= -ltls DPADD= ${LIBTLS} -REGRESS_TARGETS= run-turn +REGRESS_TARGETS= run-turn run-agent # a pty driver to exercise the curses front end end to end (2.1) ptydrive: ${.CURDIR}/ptydrive.c @@ -14,4 +14,7 @@ ptydrive: ${.CURDIR}/ptydrive.c run-turn: stub ptydrive sh ${.CURDIR}/run.sh ${.OBJDIR} ${.CURDIR}/../.. +run-agent: stub + sh ${.CURDIR}/agent.sh ${.OBJDIR} ${.CURDIR}/../.. + .include blob - /dev/null blob + 108c066fffece98abe222353b3520c7fbb28d169 (mode 644) --- /dev/null +++ regress/turn/agent.sh @@ -0,0 +1,727 @@ +#!/bin/sh +# +# Copyright (c) 2026 Isaac +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +# End-to-end M9 subagent regression. This runs the real worker mesh against +# the turn suite's TLS provider stub and proves the fresh conversation, +# read-only tool surface, result-only return boundary, progress, journal +# boundary, and exact model-palette enforcement. +# Usage: agent.sh + +set -u + +obj=$1 +root=$2 +checks=0 +fails=0 + +pass() { checks=$((checks + 1)); echo "ok $checks - $1"; } +fail() { + checks=$((checks + 1)) + fails=$((fails + 1)) + echo "FAIL $checks - $1" +} +ok() { if [ "$2" -eq 0 ]; then pass "$1"; else fail "$1"; fi; } + +dir=$(mktemp -d /tmp/fugu-agent.XXXXXXXXXX) +stubpid='' +fugupid='' +cleanup() { + [ -n "$fugupid" ] && { + kill -TERM "$fugupid" 2>/dev/null + wait "$fugupid" 2>/dev/null + } + [ -n "$stubpid" ] && { + kill "$stubpid" 2>/dev/null + wait "$stubpid" 2>/dev/null + } + rm -rf "$dir" +} +trap cleanup EXIT INT TERM + +openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 \ + -keyout "$dir/key.pem" -out "$dir/cert.pem" -days 2 -nodes \ + -subj "/CN=127.0.0.1" -addext "subjectAltName=IP:127.0.0.1" \ + >/dev/null 2>&1 + +lx=$dir/libexec/fugu +mkdir -p "$lx" +for w in fugu-tty fugu-api fugu-web fugu-tool fugu-editor; do + b=$root/src/$w/obj/$w + [ -x "$b" ] || b=$root/src/$w/$w + [ -x "$b" ] || { echo "FAIL - $w not built"; exit 1; } + cp "$b" "$lx/$w" +done +fugu=$root/src/fugu/obj/fugu +[ -x "$fugu" ] || fugu=$root/src/fugu/fugu + +export HOME=$dir/home +mkdir -p "$HOME" "$dir/work" +sessions=$HOME/.fugu/sessions +KEY=sk-agent-test-key + +start_stub() { + rm -f "$dir/port" "$dir/reqout" "$dir/reqlog" "$dir/keyout" \ + "$dir/overlap" "$dir/pool" "$dir/weborder" + KEYOUT=$dir/keyout REQOUT=$dir/reqout REQLOG=$dir/reqlog \ + OVERLAPOUT=$dir/overlap POOLOUT=$dir/pool \ + WEBORDEROUT=$dir/weborder FUTUREPATH="$dir/work/future*" \ + "$obj/stub" -c "$dir/cert.pem" -k "$dir/key.pem" "$@" \ + >"$dir/port" 2>/dev/null & + stubpid=$! + i=0 + while [ ! -s "$dir/port" ] && [ "$i" -lt 40 ]; do + sleep 0.25 + i=$((i + 1)) + done + stubport=$(sed -n 's/^PORT //p' "$dir/port") +} + +stop_stub() { + [ -n "$stubpid" ] && { + kill "$stubpid" 2>/dev/null + wait "$stubpid" 2>/dev/null + } + stubpid='' +} + +write_conf() { + cap=${1:-1} + umask 077 + cat > "$dir/fugu.conf" <"$dir/out" 2>"$dir/errout" + rc=$? +} + +request_section() { + awk -v name="$1" ' + $0 == "--- " name " ---" { found = 1; next } + found && /^--- .* ---$/ { exit } + found { print } + ' "$dir/reqlog" > "$2" +} + +# A lead agent delegates, its child reads a local fixture and reports, and the +# lead receives just that final report before synthesizing the user response. +rm -rf "$HOME/.fugu" +mkdir -p "$dir/work/protected" +printf 'CHILD-PRIVATE-TOOL-RESULT\n' > "$dir/work/agent-fixture.txt" +printf 'CHILD-PROTECTED-SECRET\n' > "$dir/work/protected/secret.txt" +# Keep the expanded protect set larger than one imsg frame. The persistent +# executor consumes it first and each later -r worker receives the same +# descriptor-backed snapshot independently; the protected read below proves +# that the large set was neither truncated nor lost through a shared offset. +pad=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +i=0 +while [ "$i" -lt 80 ]; do + : > "$dir/work/protected/pad-$i-$pad.txt" + i=$((i + 1)) +done +start_stub AGENTCALL AGENTPROTECT AGENTWRITE AGENTNET AGENTREAD AGENTREPORT \ + AGENTSYNTH +write_conf +run_fugu "LEAD-ONLY-PROMPT delegate this inspection" +ok "agent loop: exit 0" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'LEAD-SYNTHESIS-MARKER' "$dir/out" +ok "agent loop: lead synthesis reaches stdout" $? +grep -q 'CHILD-REPORT-MARKER' "$dir/out" 2>/dev/null +ok "agent loop: child report is not printed as the answer" \ + $([ $? -ne 0 ]; echo $?) + +request_section AGENTCALL "$dir/lead-initial" +request_section AGENTPROTECT "$dir/child-initial" +request_section AGENTWRITE "$dir/child-protected" +request_section AGENTNET "$dir/child-write" +request_section AGENTREAD "$dir/child-net" +request_section AGENTREPORT "$dir/child-followup" +request_section AGENTSYNTH "$dir/lead-followup" + +grep -q '"name":"agent"' "$dir/lead-initial" +ok "agent loop: lead is offered the agent tool" $? +grep -q 'AGENT-FRESH-PROMPT' "$dir/child-initial" +ok "agent loop: delegated prompt starts the child conversation" $? +grep -q 'LEAD-ONLY-PROMPT' "$dir/child-initial" 2>/dev/null +ok "agent loop: lead conversation is absent from the child" \ + $([ $? -ne 0 ]; echo $?) + +readonly_ok=0 +for name in read shell grep find ls web_search web_fetch; do + grep -q "\"name\":\"$name\"" "$dir/child-initial" || readonly_ok=1 +done +ok "agent loop: child receives the complete read-only tool surface" \ + "$readonly_ok" +forbidden=0 +for name in edit write shell_bg shell_output agent skill http_request; do + if grep -q "\"name\":\"$name\"" "$dir/child-initial"; then + forbidden=1 + fi +done +ok "agent loop: child receives no mutating, background, or nested tools" \ + "$forbidden" + +grep -q 'protected/secret.txt' "$dir/child-protected" && \ + grep -q '"is_error":true' "$dir/child-protected" +ok "agent loop: startup-protected path is refused in the child" $? +grep -q 'CHILD-PROTECTED-SECRET' "$dir/reqlog" 2>/dev/null +ok "agent loop: protected bytes never leave the ephemeral worker" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'agent-write-attempt' "$dir/child-write" && \ + grep -q '"is_error":true' "$dir/child-write" +ok "agent loop: shell mutation is refused by the read-only child" $? +test ! -e "$dir/work/agent-write-attempt" +ok "agent loop: denied shell mutation creates no file" $? +grep -q 'SUBAGENT-NET-DENIED' "$dir/child-net" && \ + grep -q '"is_error":true' "$dir/child-net" +ok "agent loop: child network is denied despite lead opt-in" $? +grep -q '"content":"SUBAGENT-NET-OPEN' "$dir/child-net" 2>/dev/null +ok "agent loop: child never establishes the forbidden connection" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'CHILD-PRIVATE-TOOL-RESULT' "$dir/child-followup" +ok "agent loop: local read result reaches the child follow-up" $? +grep -q 'CHILD-REPORT-MARKER' "$dir/lead-followup" +ok "agent loop: final child report reaches the lead" $? +grep -q 'CHILD-PRIVATE-TOOL-RESULT' "$dir/lead-followup" 2>/dev/null +ok "agent loop: child internal tool result stays out of lead context" \ + $([ $? -ne 0 ]; echo $?) +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "agent loop: exactly two lead and five child requests" \ + $([ "$n" -eq 7 ]; echo $?) + +grep -q 'agent\[reader\]: started' "$dir/errout" +ok "agent loop: labelled start progress is visible" $? +grep -q 'agent\[reader\]: tool read' "$dir/errout" +ok "agent loop: labelled tool progress is visible" $? +grep -q 'agent\[reader\]: completed' "$dir/errout" +ok "agent loop: labelled completion is visible" $? + +grep -q '"name":"agent"' "$sessions"/*.ndjson 2>/dev/null +ok "agent loop: lead spawn is journaled" $? +grep -q 'CHILD-REPORT-MARKER' "$sessions"/*.ndjson 2>/dev/null +ok "agent loop: final child report is journaled as the result" $? +grep -q 'CHILD-PRIVATE-TOOL-RESULT' "$sessions"/*.ndjson 2>/dev/null +ok "agent loop: child local transcript is not journaled" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'CHILD-PROTECTED-SECRET' "$sessions"/*.ndjson 2>/dev/null +ok "agent loop: protected bytes are absent from the journal" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'ephemeral read-only subagent' "$sessions"/*.ndjson 2>/dev/null +ok "agent loop: child system context is not journaled" \ + $([ $? -ne 0 ]; echo $?) +grep -q '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null +ok "agent loop: the lead turn commits" $? +grep -q "$KEY" "$dir/keyout" 2>/dev/null +ok "agent loop: provider traffic stays credential-brokered" $? +stop_stub + +# An escaped glob metacharacter is a literal future path, not an unmatched +# pattern. It is absent when the coordinator snapshots protect, then the +# provider stub creates the literal-star filename before the child starts. +rm -rf "$HOME/.fugu" +rm -f "$dir/work/future*" +start_stub FUTURECALL FUTUREREAD FUTUREREPORT FUTURESYNTH +write_conf +test ! -e "$dir/work/future*" +ok "future literal: escaped path is absent at startup" $? +run_fugu "inspect a future literal protected path" +ok "future literal: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +test -f "$dir/work/future*" +ok "future literal: fixture is created only after startup" $? +grep -q 'FUTURE-LITERAL-SYNTHESIS-MARKER' "$dir/out" +ok "future literal: lead synthesis reaches stdout" $? +request_section FUTUREREPORT "$dir/future-child-followup" +grep -Fq 'future*' "$dir/future-child-followup" && \ + grep -q '"is_error":true' "$dir/future-child-followup" +ok "future literal: unescaped literal path remains protected" $? +grep -q 'FUTURE-LITERAL-SECRET' "$dir/reqlog" 2>/dev/null +ok "future literal: protected bytes never reach the provider" \ + $([ $? -ne 0 ]; echo $?) +request_section FUTURESYNTH "$dir/future-lead-followup" +grep -q 'FUTURE-LITERAL-PROTECTED-REPORT' "$dir/future-lead-followup" +ok "future literal: protected diagnostic returns to the lead" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "future literal: exactly two child and two lead requests ran" \ + $([ "$n" -eq 4 ]; echo $?) +stop_stub + +# A provider-supplied label is terminal-visible, so the coordinator validates +# it again at execution time even if the provider ignored the JSON schema. +rm -rf "$HOME/.fugu" +start_stub AGENTLABEL AGENTSYNTH +write_conf +run_fugu "try a hostile subagent progress label" +ok "invalid label: lead turn still completes" $([ "$rc" -eq 0 ]; echo $?) +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "invalid label: no child provider request was made" \ + $([ "$n" -eq 2 ]; echo $?) +request_section AGENTSYNTH "$dir/label-followup" +grep -q 'label must be a short ASCII identifier' "$dir/label-followup" +ok "invalid label: execution-time error returns to the lead" $? +grep -q 'agent\[bad' "$dir/errout" 2>/dev/null +ok "invalid label: no attacker-controlled progress was emitted" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'label must be a short ASCII identifier' \ + "$sessions"/*.ndjson 2>/dev/null +ok "invalid label: rejected result follows normal journaling" $? +stop_stub + +# Even when a provider ignores the advertised enum, an exact palette miss is +# returned to the lead as a tool error without launching a child request. +rm -rf "$HOME/.fugu" +start_stub AGENTBAD AGENTSYNTH +write_conf +run_fugu "try a forbidden subagent route" +ok "invalid route: lead turn still completes" $([ "$rc" -eq 0 ]; echo $?) +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "invalid route: no child provider request was made" \ + $([ "$n" -eq 2 ]; echo $?) +request_section AGENTSYNTH "$dir/bad-followup" +grep -q 'outside the configured palette' "$dir/bad-followup" +ok "invalid route: exact palette error returns to the lead" $? +grep -q 'agent\[blocked\]' "$dir/errout" 2>/dev/null +ok "invalid route: no child progress was emitted" $([ $? -ne 0 ]; echo $?) +grep -q 'outside the configured palette' "$sessions"/*.ndjson 2>/dev/null +ok "invalid route: rejected result follows normal journaling" $? +stop_stub + +# Two calls with a one-slot cap must run as separate waves. Their results are +# still returned to the lead in the original tool-call order. +rm -rf "$HOME/.fugu" +start_stub WAVECALL WAVEA WAVEB WAVESYNTH +write_conf +run_fugu "run two inspections with one subagent slot" +ok "wave cap: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'WAVE-SYNTHESIS-MARKER' "$dir/out" +ok "wave cap: lead synthesis reaches stdout" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "wave cap: two children and one lead follow-up ran" \ + $([ "$n" -eq 4 ]; echo $?) +request_section WAVEA "$dir/wave-a" +request_section WAVEB "$dir/wave-b" +request_section WAVESYNTH "$dir/wave-lead" +grep -q 'WAVE-PROMPT-A' "$dir/wave-a" +ok "wave cap: first child receives only its task" $? +grep -q 'WAVE-PROMPT-B' "$dir/wave-a" 2>/dev/null +ok "wave cap: second task is absent from the first child" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'WAVE-PROMPT-B' "$dir/wave-b" +ok "wave cap: queued child receives the second task" $? +a_done=$(grep -n 'agent\[wave-a\]: completed' "$dir/errout" | \ + head -1 | cut -d: -f1) +b_start=$(grep -n 'agent\[wave-b\]: started' "$dir/errout" | \ + head -1 | cut -d: -f1) +ordered=1 +if [ -n "$a_done" ] && [ -n "$b_start" ] && [ "$a_done" -lt "$b_start" ]; then + ordered=0 +fi +ok "wave cap: slot one completes before queued slot two starts" "$ordered" +grep -q 'WAVE-REPORT-A.*WAVE-REPORT-B' "$dir/wave-lead" +ok "wave cap: results return to the lead in call order" $? +grep -q 'WAVE-REPORT-A' "$sessions"/*.ndjson 2>/dev/null && \ + grep -q 'WAVE-REPORT-B' "$sessions"/*.ndjson 2>/dev/null +ok "wave cap: both result boundaries are journaled" $? +stop_stub + +# With two slots, both provider requests must arrive before either is +# answered. The stub then completes them in reverse arrival order; the lead +# must still receive each labelled result in original tool-call order. +rm -rf "$HOME/.fugu" +start_stub WAVECALL OVERLAPA OVERLAPB OVERLAPSYNTH +write_conf 2 +run_fugu "run two subagents concurrently" +ok "overlap: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'OVERLAP-SYNTHESIS-MARKER' "$dir/out" +ok "overlap: lead synthesis reaches stdout" $? +test -s "$dir/overlap" +ok "overlap: two provider requests were in flight together" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "overlap: exactly two child requests and two lead requests ran" \ + $([ "$n" -eq 4 ]; echo $?) +request_section OVERLAPSYNTH "$dir/overlap-lead" +grep -q 'OVERLAP-REPORT-A.*OVERLAP-REPORT-B' "$dir/overlap-lead" +ok "overlap: reverse completions correlate back into call order" $? +grep -q 'agent\[wave-a\]: started' "$dir/errout" && \ + grep -q 'agent\[wave-b\]: started' "$dir/errout" && \ + grep -q 'agent\[wave-a\]: completed' "$dir/errout" && \ + grep -q 'agent\[wave-b\]: completed' "$dir/errout" +ok "overlap: both labelled agents expose live progress" $? +grep -q 'OVERLAP-REPORT-A.*OVERLAP-REPORT-B' \ + "$sessions"/*.ndjson 2>/dev/null +ok "overlap: correlated result order is journaled" $? +stop_stub + +# Both children reach web_fetch concurrently, but the single broker must not +# overlap their outbound requests. The origin holds the first response while +# checking that no second connection is pending, then releases both results. +rm -rf "$HOME/.fugu" +start_stub WAVECALL WEBPROVA WEBPROVB WEBREPORT WEBREPORT WEBSYNTH +write_conf 2 +export FUGU_FETCH_ALLOW_PRIVATE=1 +run_fugu "run two concurrent subagent web inspections" +unset FUGU_FETCH_ALLOW_PRIVATE +ok "web serialization: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'WEB-SYNTHESIS-MARKER' "$dir/out" +ok "web serialization: lead synthesis reaches stdout" $? +test -s "$dir/overlap" +ok "web serialization: both child provider calls overlap" $? +serial=0 +webcount=0 +first=? +second=? +if [ -s "$dir/weborder" ]; then + read serial webcount first second < "$dir/weborder" +fi +weborder_ok=1 +case "$first$second" in +ab|ba) + [ "$serial" -eq 1 ] && [ "$webcount" -eq 2 ] && weborder_ok=0 + ;; +esac +ok "web serialization: broker issues two origin requests serially" \ + "$weborder_ok" +n=$(grep -c '^--- WEBREPORT ---$' "$dir/reqlog") +ok "web serialization: both broker results reach child follow-ups" \ + $([ "$n" -eq 2 ] && \ + grep -q 'WEB-MESH-RESULT-A' "$dir/reqlog" && \ + grep -q 'WEB-MESH-RESULT-B' "$dir/reqlog"; echo $?) +request_section WEBSYNTH "$dir/web-lead-followup" +grep -q 'WEB-REPORT-A.*WEB-REPORT-B' "$dir/web-lead-followup" +ok "web serialization: reports return to the lead in call order" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "web serialization: exactly two child web rounds complete" \ + $([ "$n" -eq 6 ]; echo $?) +grep -q 'agent\[wave-a\]: completed' "$dir/errout" && \ + grep -q 'agent\[wave-b\]: completed' "$dir/errout" +ok "web serialization: both labelled children complete" $? +stop_stub + +# Retry backoff owns no live socket. Fill the custodian's eight network +# slots with distinct 503 responses carrying Retry-After: 2; the ninth child +# must be admitted and complete before any of those eight timers can fire. +rm -rf "$HOME/.fugu" +set -- POOLCALL +i=0 +while [ "$i" -lt 8 ]; do + set -- "$@" POOL503 + i=$((i + 1)) +done +set -- "$@" POOLWIN +i=0 +while [ "$i" -lt 8 ]; do + set -- "$@" POOLBACK + i=$((i + 1)) +done +set -- "$@" POOLSYNTH +start_stub "$@" +write_conf 9 +run_fugu "run nine correlated subagents through the provider pool" +ok "pool: lead turn completes after all correlated retries" \ + $([ "$rc" -eq 0 ]; echo $?) +grep -q 'POOL-SYNTHESIS-MARKER' "$dir/out" +ok "pool: lead synthesis reaches stdout" $? +fresh=0 +seen=0 +elapsed=999999 +if [ -s "$dir/pool" ]; then + read fresh seen elapsed < "$dir/pool" +fi +[ "$fresh" -eq 1 ] && [ "$seen" -eq 8 ] +ok "pool: ninth request is distinct from all eight backoff requests" $? +[ "$elapsed" -ge 0 ] && [ "$elapsed" -lt 1800 ] +ok "pool: ninth request completes before Retry-After can expire" $? +n503=$(grep -c '^--- POOL503 ---$' "$dir/reqlog") +nback=$(grep -c '^--- POOLBACK ---$' "$dir/reqlog") +nretry=$(grep -c 'provider retry' "$dir/errout") +ok "pool: exactly eight requests back off and retry once" \ + $([ "$n503" -eq 8 ] && [ "$nback" -eq 8 ] && \ + [ "$nretry" -eq 8 ]; echo $?) +stop_stub + +# A token-limited text reply is not mistaken for a completed report. Its +# partial text is preserved, followed by one tool-free conclusion request. +rm -rf "$HOME/.fugu" +start_stub AGENTCALL LIMITTEXT LIMITCONCLUDE LIMITSYNTH +write_conf +run_fugu "exercise a text-only subagent token limit" +ok "token limit: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'LIMIT-SYNTHESIS-MARKER' "$dir/out" +ok "token limit: lead synthesis reaches stdout" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "token limit: exactly one conclusion generation ran" \ + $([ "$n" -eq 4 ]; echo $?) +request_section LIMITCONCLUDE "$dir/limit-text-conclusion" +grep -q 'LIMIT-TEXT-PARTIAL' "$dir/limit-text-conclusion" +ok "token limit: partial text survives into the conclusion" $? +sed -n '/^{/s/,"messages":.*//p' "$dir/limit-text-conclusion" \ + > "$dir/limit-text-prefix" +grep -q '"tools"' "$dir/limit-text-prefix" 2>/dev/null +ok "token limit: conclusion request offers no tools" \ + $([ $? -ne 0 ]; echo $?) +request_section LIMITSYNTH "$dir/limit-text-lead" +grep -q 'LIMIT-TEXT-PARTIAL' "$dir/limit-text-lead" && \ + grep -q 'LIMIT-CONCLUSION-TEXT' "$dir/limit-text-lead" +ok "token limit: partial and conclusion text return to the lead" $? +grep -q 'subagent report truncated by its execution bound' \ + "$dir/limit-text-lead" +ok "token limit: final report is marked truncated" $? +grep -q 'agent\[reader\]: completed (truncated)' "$dir/errout" +ok "token limit: progress marks truncation" $? +stop_stub + +# A length stop may arrive alongside a tool_use. The tool is left unexecuted +# but receives an error result so the one tool-free conclusion projection is +# structurally valid and retains the partial assistant text. +rm -rf "$HOME/.fugu" +start_stub AGENTCALL LIMITCALL LIMITCONCLUDE LIMITSYNTH +write_conf +run_fugu "exercise a dangling tool at the subagent token limit" +ok "length limit: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "length limit: exactly one conclusion generation ran" \ + $([ "$n" -eq 4 ]; echo $?) +request_section LIMITCONCLUDE "$dir/limit-call-conclusion" +grep -q 'LIMIT-CALL-PARTIAL' "$dir/limit-call-conclusion" +ok "length limit: partial text survives into the conclusion" $? +grep -q '"tool_use_id":"toolu_dangle"' \ + "$dir/limit-call-conclusion" && \ + grep -q 'truncated and was not executed' "$dir/limit-call-conclusion" && \ + grep -q '"is_error":true' "$dir/limit-call-conclusion" +ok "length limit: dangling tool receives an error result" $? +sed -n '/^{/s/,"messages":.*//p' "$dir/limit-call-conclusion" \ + > "$dir/limit-call-prefix" +grep -q '"tools"' "$dir/limit-call-prefix" 2>/dev/null +ok "length limit: conclusion request offers no tools" \ + $([ $? -ne 0 ]; echo $?) +grep -q 'agent\[reader\]: tool read' "$dir/errout" 2>/dev/null +ok "length limit: dangling tool is never executed" $([ $? -ne 0 ]; echo $?) +request_section LIMITSYNTH "$dir/limit-call-lead" +grep -q 'LIMIT-CALL-PARTIAL' "$dir/limit-call-lead" && \ + grep -q 'LIMIT-CONCLUSION-TEXT' "$dir/limit-call-lead" && \ + grep -q 'subagent report truncated' "$dir/limit-call-lead" +ok "length limit: bounded report preserves all available text" $? +stop_stub + +# Retained interim prose may itself exceed the 1 MiB tool-result boundary. +# The bounded projection must keep a useful head and the later, much smaller +# diagnostic conclusion rather than trimming the conclusion off the end. +rm -rf "$HOME/.fugu" +start_stub AGENTCALL RETAINLIMIT RETAINCONCLUDE RETAINSYNTH +write_conf +run_fugu "exercise a large retained subagent report" +ok "retained report: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'RETAIN-SYNTHESIS-MARKER' "$dir/out" +ok "retained report: lead synthesis reaches stdout" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "retained report: exactly one conclusion generation ran" \ + $([ "$n" -eq 4 ]; echo $?) +request_section RETAINCONCLUDE "$dir/retain-conclusion" +retain_conclusion_bytes=$(wc -c < "$dir/retain-conclusion") +grep -q 'RETAINED-PROSE-HEAD' "$dir/retain-conclusion" && \ + [ "$retain_conclusion_bytes" -gt 1500000 ] +ok "retained report: more than 1 MiB survives into conclusion" $? +sed -n '/^{/s/,"messages":.*//p' "$dir/retain-conclusion" \ + > "$dir/retain-prefix" +grep -q '"tools"' "$dir/retain-prefix" 2>/dev/null +ok "retained report: conclusion request offers no tools" \ + $([ $? -ne 0 ]; echo $?) +request_section RETAINSYNTH "$dir/retain-lead" +grep -q 'RETAINED-PROSE-HEAD' "$dir/retain-lead" && \ + grep -q 'RETAIN-FINAL-CONCLUSION-DIAGNOSTIC' "$dir/retain-lead" && \ + grep -q 'subagent report truncated by its execution bound' \ + "$dir/retain-lead" +ok "retained report: cap preserves head, diagnostic tail, and marker" $? +retain_lead_bytes=$(wc -c < "$dir/retain-lead") +[ "$retain_lead_bytes" -gt 1000000 ] && \ + [ "$retain_lead_bytes" -lt 1200000 ] +ok "retained report: lead projection stays within its 1 MiB bound" $? +grep -q 'agent\[reader\]: completed (truncated)' "$dir/errout" +ok "retained report: progress marks truncation" $? +stop_stub + +# The same head/tail rule applies to terminal failures after useful interim +# prose. A tool-bearing generation retains 1.5 MiB, then the next provider +# generation fails; the bounded error returned to the lead must keep both. +rm -rf "$HOME/.fugu" +start_stub AGENTCALL RETAINTOOL ERROR RETAINERRSYNTH +write_conf +run_fugu "exercise a terminal error after a large retained report" +ok "retained error: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'RETAIN-ERROR-SYNTHESIS-MARKER' "$dir/out" +ok "retained error: lead synthesis reaches stdout" $? +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "retained error: exactly one tool round precedes the failure" \ + $([ "$n" -eq 4 ]; echo $?) +request_section ERROR "$dir/retain-error-request" +retain_error_request_bytes=$(wc -c < "$dir/retain-error-request") +grep -q 'RETAINED-PROSE-HEAD' "$dir/retain-error-request" && \ + grep -q 'CHILD-PRIVATE-TOOL-RESULT' "$dir/retain-error-request" && \ + [ "$retain_error_request_bytes" -gt 1500000 ] +ok "retained error: large interim prose reaches the failing generation" $? +request_section RETAINERRSYNTH "$dir/retain-error-lead" +grep -q 'RETAINED-PROSE-HEAD' "$dir/retain-error-lead" && \ + grep -q 'subagent provider error' "$dir/retain-error-lead" && \ + grep -q 'stub overloaded' "$dir/retain-error-lead" && \ + grep -q 'subagent report truncated by its execution bound' \ + "$dir/retain-error-lead" +ok "retained error: cap preserves head, terminal diagnostic, and marker" $? +retain_error_lead_bytes=$(wc -c < "$dir/retain-error-lead") +[ "$retain_error_lead_bytes" -gt 1000000 ] && \ + [ "$retain_error_lead_bytes" -lt 1200000 ] +ok "retained error: failed result remains within its 1 MiB bound" $? +grep -q 'agent\[reader\]: completed (truncated)' "$dir/errout" +ok "retained error: failed completion reports truncation" $? +stop_stub + +# Tool-input bytes and emitted tool results both participate in the same +# 4 MiB child aggregate. Each hostile case fails locally after its terminal, +# without starting an extra provider generation or crashing the worker mesh. +rm -rf "$HOME/.fugu" +start_stub AGENTCALL ARGBOUND LIMITSYNTH +write_conf +run_fugu "exercise the subagent tool-input aggregate" +ok "input aggregate: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "input aggregate: no extra child generation ran" \ + $([ "$n" -eq 3 ]; echo $?) +request_section LIMITSYNTH "$dir/input-bound-lead" +grep -q 'subagent data bound exceeded' "$dir/input-bound-lead" +ok "input aggregate: oversized input returns a bounded local error" $? +stop_stub + +rm -rf "$HOME/.fugu" +start_stub AGENTCALL RESULTBOUND LIMITSYNTH +write_conf +run_fugu "exercise the subagent result aggregate" +ok "result aggregate: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +n=$(grep -c '^--- .* ---$' "$dir/reqlog") +ok "result aggregate: no extra child generation ran" \ + $([ "$n" -eq 3 ]; echo $?) +request_section LIMITSYNTH "$dir/result-bound-lead" +grep -q 'subagent data bound exceeded' "$dir/result-bound-lead" +ok "result aggregate: emitted results return a bounded local error" $? +stop_stub + +# A child gets 25 tool-bearing generations, then exactly one final generation +# with no offered tools. The bounded report remains useful and conspicuous. +rm -rf "$HOME/.fugu" +set -- AGENTCALL +i=0 +while [ "$i" -lt 25 ]; do + set -- "$@" AGENTROUND + i=$((i + 1)) +done +set -- "$@" AGENTCONCLUDE BOUNDSYNTH +start_stub "$@" +write_conf +run_fugu "exercise the bounded subagent loop" +ok "round bound: lead turn completes" $([ "$rc" -eq 0 ]; echo $?) +grep -q 'BOUND-SYNTHESIS-MARKER' "$dir/out" +ok "round bound: lead synthesis reaches stdout" $? +n=$(grep -c '^--- AGENTROUND ---$' "$dir/reqlog") +ok "round bound: exactly 25 tool-bearing generations ran" \ + $([ "$n" -eq 25 ]; echo $?) +request_section AGENTCONCLUDE "$dir/bound-conclusion" +grep -q 'reached the subagent tool-round bound' "$dir/bound-conclusion" +ok "round bound: the conclusion instruction is injected" $? +sed -n '/^{/s/,"messages":.*//p' "$dir/bound-conclusion" \ + > "$dir/bound-prefix" +grep -q '"tools"' "$dir/bound-prefix" 2>/dev/null +ok "round bound: the conclusion request offers no tools" \ + $([ $? -ne 0 ]; echo $?) +request_section BOUNDSYNTH "$dir/bound-lead" +grep -q 'BOUND-REPORT-MARKER' "$dir/bound-lead" +ok "round bound: conclusion report returns to the lead" $? +grep -q 'subagent report truncated by its execution bound' "$dir/bound-lead" +ok "round bound: lead sees the conspicuous truncation marker" $? +grep -q 'agent\[reader\]: completed (truncated)' "$dir/errout" +ok "round bound: completion progress marks truncation" $? +n=$(grep -c 'agent\[reader\]: tool read' "$dir/errout") +ok "round bound: progress records all 25 bounded tool rounds" \ + $([ "$n" -eq 25 ]; echo $?) +stop_stub + +# SIGINT abandons a turn and must kill both the ephemeral worker and its +# currently active local process group. A watchdog keeps a regression from +# hanging forever if cancellation breaks. +rm -rf "$HOME/.fugu" +rm -f "$dir/cancel-timeout" +start_stub AGENTCALL AGENTHANG AGENTREPORT AGENTSYNTH +write_conf +( + cd "$dir/work" || exit 1 + exec env FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \ + FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \ + "$fugu" -p "start an agent that will be cancelled" "$dir/out" 2> "$dir/errout" & +fugupid=$! +i=0 +active=1 +while [ "$i" -lt 50 ]; do + ps -axww -o command > "$dir/ps-before" + if grep -q '[A]GENT-CANCEL-MARKER' "$dir/ps-before"; then + active=0 + break + fi + sleep 0.1 + i=$((i + 1)) +done +ok "cancellation: subagent local child was active" "$active" +kill -INT "$fugupid" 2>/dev/null +( + sleep 5 + if kill -0 "$fugupid" 2>/dev/null; then + : > "$dir/cancel-timeout" + kill -TERM "$fugupid" 2>/dev/null + fi +) & +watchpid=$! +wait "$fugupid" +cancelrc=$? +fugupid='' +kill "$watchpid" 2>/dev/null +wait "$watchpid" 2>/dev/null +test ! -e "$dir/cancel-timeout" +ok "cancellation: coordinator exits promptly" $? +ok "cancellation: print mode reports cancellation status" \ + $([ "$cancelrc" -eq 3 ]; echo $?) +sleep 0.2 +ps -axww -o command > "$dir/ps-after" +grep -q '[A]GENT-CANCEL-MARKER' "$dir/ps-after" 2>/dev/null +ok "cancellation: active local process group is gone" \ + $([ $? -ne 0 ]; echo $?) +grep -q '"outcome":"abandoned"' "$sessions"/*.ndjson 2>/dev/null +ok "cancellation: journal records an abandoned turn" $? +grep -q '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null +ok "cancellation: cancelled turn does not commit" $([ $? -ne 0 ]; echo $?) +stop_stub + +echo "$checks checks, $fails failures" +[ "$fails" -eq 0 ] blob - 7699ebd16a0d25eff789f3cc9f1b2152014ff7e0 blob + af472ed1cc518e7f12dd4011291cb0df807503d3 --- regress/turn/run.sh +++ regress/turn/run.sh @@ -752,18 +752,39 @@ if [ -x "$obj/ptydrive" ]; then rm -f "$dir/reqout" start_stub "OK" write_conf "$stubport" - printf '/model\r\000\000model-b\r\000hello\r\000/quit\r' | \ + printf '/model\r\000\000model-b\r\000/context\r\000hello\r\000/quit\r' | \ TERM=vt100 HOME=$HOME \ FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \ FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \ - "$obj/ptydrive" "$fugu" >/dev/null 2>&1 + "$obj/ptydrive" "$fugu" >"$dir/pickerout" 2>&1 rc=$? ok "picker: clean exit" $([ "$rc" -eq 0 ]; echo $?) grep -q '"model":"stub-model-b"' "$dir/reqout" 2>/dev/null ok "picker: the selected model drove the next request" $? + grep -a -q "424242" "$dir/pickerout" 2>/dev/null + ok "picker: selected provider metadata drove the context window" $? stop_stub + + # An explicit context_limit remains the first-priority denominator. + rm -rf "$HOME/.fugu" + start_stub "OK" + write_conf "$stubport" + printf 'context_limit 777777\n' >> "$dir/fugu.conf" + printf '/model\r\000\000model-b\r\000/context\r\000/quit\r' | \ + TERM=vt100 HOME=$HOME \ + FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \ + FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \ + "$obj/ptydrive" "$fugu" >"$dir/pickerout" 2>&1 + rc=$? + ok "picker context override: clean exit" $([ "$rc" -eq 0 ]; echo $?) + grep -a -q "777777" "$dir/pickerout" 2>/dev/null + ok "picker context override: context_limit wins" $? + stop_stub else fail "picker: ptydrive helper not built" + fail "picker: selected provider metadata drove the context window" + fail "picker context override: clean exit" + fail "picker context override: context_limit wins" fi # --- 32: the drawer switches sessions by re-exec (behavior.md 2.1) --- blob - fc43b8bc2c324cf992d492b1be8be1161741cd6d blob + 9697df6b559601e6b8fc3a9031bb897f6d7d91ec --- regress/turn/stub.c +++ regress/turn/stub.c @@ -25,7 +25,25 @@ * stream with no content blocks), WEBFETCH (a brokered web_fetch), * BGTOOL (shell_bg sleep 300), ERROR_BG (the provider failure after * that launch), JOBOUTPUT (shell_output job 1), DOUBLE (a hostile error event - * appended after a complete OK stream). Prints "PORT n" once + * appended after a complete OK stream), AGENTCALL (lead delegates to a + * labelled subagent), AGENTREAD (the subagent reads a fixture), AGENTREPORT + * AGENTPROTECT (a read of a startup-protected path), AGENTWRITE (a denied + * shell mutation), AGENTNET (a denied network connection), AGENTREPORT (the + * subagent's final report), AGENTSYNTH, AGENTBAD (a hostile out-of-palette + * route), AGENTLABEL (a hostile terminal-control label), + * WAVECALL/WAVEA/WAVEB/WAVESYNTH + * (two calls queued through a one-slot wave), AGENTROUND/AGENTCONCLUDE/ + * BOUNDSYNTH (the 25-round bound and tool-free conclusion), and AGENTHANG + * (cancellation of an active local child). OVERLAPA/OVERLAPB hold two + * requests in flight and complete them in reverse order. LIMITTEXT/LIMITCALL + * exercise token stops; RETAINLIMIT/RETAINCONCLUDE and RETAINTOOL/ERROR + * preserve a capped report's conclusion and terminal diagnostic tails; + * ARGBOUND/RESULTBOUND exercise child data accounting; + * WEBPROVA/WEBPROVB/WEBREPORT prove concurrent children are serialized through + * the web broker; FUTURECALL/FUTUREREAD prove an escaped literal protect path; + * POOLCALL/POOL503/POOLWIN/POOLBACK/POOLSYNTH prove retry backoff releases + * one of the API custodian's eight live-network slots. + * Prints "PORT n" once * listening; writes the request's x-api-key value to the file named * by KEYOUT and the whole request to the file named by REQOUT so the * test can prove what arrived. @@ -35,13 +53,19 @@ #include #include +#include #include #include #include #include +#include #include #include +#define STUB_TURN_DATA_MAX (4U * 1024 * 1024) +#define STUB_REQ_MAX (2U * 1024 * 1024) +#define STUB_RETAIN_TEXT (1536U * 1024) + static const char body_ok[] = "HTTP/1.1 200 OK\r\n" "content-type: text/event-stream\r\n" @@ -116,13 +140,545 @@ static const char sse_tooluse[] = "event: message_stop\n" "data: {\"type\":\"message_stop\"}\n\n"; +/* The lead delegates a self-contained task to a labelled subagent. */ +static const char sse_agentcall[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":13,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_agent\"," + "\"name\":\"agent\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"prompt\\\":\\\"Inspect agent-fixture.txt and report " + "AGENT-FRESH-PROMPT.\\\",\\\"label\\\":\\\"reader\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":14}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* Create-after-startup coverage for an escaped literal protect token. */ +static const char sse_futurecall[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":13,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_future\"," + "\"name\":\"agent\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"prompt\\\":\\\"Read the literal future* path and report.\\\"," + "\\\"label\\\":\\\"future\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":14}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* The fresh subagent uses one of its local read-only tools. */ +static const char sse_agentread[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":17,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_read\"," + "\"name\":\"read\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"path\\\":\\\"agent-fixture.txt\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* The protected path must stay subtracted in every dynamic worker. */ +static const char sse_agentprotect[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":17,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_protect\"," + "\"name\":\"read\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"path\\\":\\\"protected/secret.txt\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +static const char sse_futureread[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":17,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_future_read\"," + "\"name\":\"read\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"path\\\":\\\"future*\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* A hostile shell redirection must fail under the child's rx unveil. */ +static const char sse_agentwrite[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":17,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_write\"," + "\"name\":\"shell\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"command\\\":\\\"printf forbidden > " + "agent-write-attempt\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* A long local child gives the cancellation regression an active process. */ +static const char sse_agenthang[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":17,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_hang\"," + "\"name\":\"shell\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"command\\\":\\\"sleep 300; : # " + "AGENT-CANCEL-MARKER\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +static const char sse_agentreport[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":21,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"text_delta\",\"text\":\"CHILD-REPORT-MARKER\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\"},\"usage\":{\"output_tokens\":7}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +static const char sse_agentsynth[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":25,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"text_delta\",\"text\":\"LEAD-SYNTHESIS-MARKER\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\"},\"usage\":{\"output_tokens\":8}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* A provider may ignore the enum; the coordinator must still reject it. */ +static const char sse_agentbad[] = +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_bad\"," + "\"name\":\"agent\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"prompt\\\":\\\"must not run\\\",\\\"label\\\":" + "\\\"blocked\\\",\\\"model\\\":\\\"not-in-palette\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* A provider may ignore the schema; labels still cannot control the tty. */ +static const char sse_agentlabel[] = +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_label\"," + "\"name\":\"agent\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"prompt\\\":\\\"must not run\\\",\\\"label\\\":" + "\\\"bad\\\\nlabel\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + +/* Two calls in one lead response exercise max_subagents wave queuing. */ +static const char sse_wavecall[] = +"event: message_start\n" +"data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":14,\"output_tokens\":1}}}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_wave_a\"," + "\"name\":\"agent\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"prompt\\\":\\\"WAVE-PROMPT-A\\\",\\\"label\\\":" + "\\\"wave-a\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" +"event: content_block_start\n" +"data: {\"type\":\"content_block_start\",\"index\":1," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_wave_b\"," + "\"name\":\"agent\"}}\n\n" +"event: content_block_delta\n" +"data: {\"type\":\"content_block_delta\",\"index\":1,\"delta\":" + "{\"type\":\"input_json_delta\",\"partial_json\":" + "\"{\\\"prompt\\\":\\\"WAVE-PROMPT-B\\\",\\\"label\\\":" + "\\\"wave-b\\\"}\"}}\n\n" +"event: content_block_stop\n" +"data: {\"type\":\"content_block_stop\",\"index\":1}\n\n" +"event: message_delta\n" +"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":16}}\n\n" +"event: message_stop\n" +"data: {\"type\":\"message_stop\"}\n\n"; + static int tls_write_all(struct tls *, const char *, size_t); static void write_chunk(struct tls *, const char *, size_t); +static void +send_text(struct tls *conn, const char *text) +{ + char sse[2048]; + int n; + + n = snprintf(sse, sizeof(sse), + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":19,\"output_tokens\":1}}}\n\n" + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\",\"text\":\"%s\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\"},\"usage\":{\"output_tokens\":7}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n", text); + if (n < 0 || (size_t)n >= sizeof(sse)) + errx(1, "text fixture too large"); + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, sse, (size_t)n); + (void)tls_write_all(conn, "0\r\n\r\n", 5); +} + +/* Nine simultaneous children exceed fugu-api's eight live-network slots. */ +static void +send_poolcall(struct tls *conn) +{ + static const char start[] = + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":23,\"output_tokens\":1}}}\n\n"; + static const char done[] = + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":27}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n"; + char event[1024]; + int i, n; + + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, start, sizeof(start) - 1); + for (i = 0; i < 9; i++) { + n = snprintf(event, sizeof(event), + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":%d," + "\"content_block\":{\"type\":\"tool_use\"," + "\"id\":\"toolu_pool_%d\",\"name\":\"agent\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":%d," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\"{\\\"prompt\\\":\\\"POOL-PROMPT-%d\\\"," + "\\\"label\\\":\\\"pool-%d\\\"}\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":%d}\n\n", + i, i, i, i, i, i); + if (n < 0 || (size_t)n >= sizeof(event)) + errx(1, "pool call fixture too large"); + write_chunk(conn, event, (size_t)n); + } + write_chunk(conn, done, sizeof(done) - 1); + (void)tls_write_all(conn, "0\r\n\r\n", 5); +} + +static void +send_limit(struct tls *conn, int with_tool) +{ + char sse[4096]; + int n; + + n = snprintf(sse, sizeof(sse), + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":19,\"output_tokens\":1}}}\n\n" + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\",\"text\":\"%s\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + "%s" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"%s\"},\"usage\":{\"output_tokens\":7}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n", + with_tool ? "LIMIT-CALL-PARTIAL" : "LIMIT-TEXT-PARTIAL", + with_tool ? + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":1," + "\"content_block\":{\"type\":\"tool_use\",\"id\":" + "\"toolu_dangle\",\"name\":\"read\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":1," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\"{\\\"path\\\":\\\"never-read\\\"}\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":1}\n\n" : "", + with_tool ? "length" : "max_tokens"); + if (n < 0 || (size_t)n >= sizeof(sse)) + errx(1, "limit fixture too large"); + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, sse, (size_t)n); + (void)tls_write_all(conn, "0\r\n\r\n", 5); +} + +/* More than one tool-result frame of useful interim prose, followed by a + * token stop or a tool round. Emit small JSON lines so the SSE decoder, + * rather than a giant fixture line, accumulates the retained report. */ +static void +send_retain(struct tls *conn, int with_tool) +{ + static const char start[] = + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":19,\"output_tokens\":1}}}\n\n" + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\"," + "\"text\":\"RETAINED-PROSE-HEAD|\"}}\n\n"; + static const char end[] = + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"max_tokens\"},\"usage\":{\"output_tokens\":7}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n"; + static const char tool_end[] = + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":1," + "\"content_block\":{\"type\":\"tool_use\",\"id\":" + "\"toolu_retain_read\",\"name\":\"read\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":1," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\"{\\\"path\\\":\\\"agent-fixture.txt\\\"}\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":1}\n\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":7}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n"; + static const char delta_prefix[] = + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\",\"text\":\""; + static const char delta_suffix[] = "\"}}\n\n"; + char event[8192]; + size_t left = STUB_RETAIN_TEXT, len, n; + + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, start, sizeof(start) - 1); + while (left > 0) { + n = left > 4096 ? 4096 : left; + memcpy(event, delta_prefix, sizeof(delta_prefix) - 1); + memset(event + sizeof(delta_prefix) - 1, 'x', n); + memcpy(event + sizeof(delta_prefix) - 1 + n, delta_suffix, + sizeof(delta_suffix) - 1); + len = sizeof(delta_prefix) - 1 + n + sizeof(delta_suffix) - 1; + write_chunk(conn, event, len); + left -= n; + } + if (with_tool) + write_chunk(conn, tool_end, sizeof(tool_end) - 1); + else + write_chunk(conn, end, sizeof(end) - 1); + (void)tls_write_all(conn, "0\r\n\r\n", 5); +} + +static void +send_argbound(struct tls *conn) +{ + static const char start[] = + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":" + "\"toolu_argbound\",\"name\":\"read\"}}\n\n"; + static const char end[] = + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":7}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n"; + static const char delta_prefix[] = + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\""; + static const char delta_suffix[] = "\"}}\n\n"; + char event[8192]; + size_t left = STUB_TURN_DATA_MAX - 1, n, len; + + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, start, sizeof(start) - 1); + while (left > 0) { + n = left > 4096 ? 4096 : left; + memcpy(event, delta_prefix, sizeof(delta_prefix) - 1); + memset(event + sizeof(delta_prefix) - 1, 'a', n); + memcpy(event + sizeof(delta_prefix) - 1 + n, delta_suffix, + sizeof(delta_suffix) - 1); + len = sizeof(delta_prefix) - 1 + n + sizeof(delta_suffix) - 1; + write_chunk(conn, event, len); + left -= n; + } + write_chunk(conn, end, sizeof(end) - 1); + (void)tls_write_all(conn, "0\r\n\r\n", 5); +} + +static void +send_resultbound(struct tls *conn) +{ + char *sse; + size_t cap = 65536, off = 0; + int i, n; + + if ((sse = calloc(1, cap)) == NULL) + errx(1, "resultbound fixture allocation"); + n = snprintf(sse, cap, + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":19,\"output_tokens\":1}}}\n\n"); + if (n < 0 || (size_t)n >= cap) + errx(1, "resultbound fixture too large"); + off = (size_t)n; + for (i = 0; i < 17; i++) { + n = snprintf(sse + off, cap - off, + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":%d," + "\"content_block\":{\"type\":\"tool_use\",\"id\":" + "\"toolu_result_%d\",\"name\":\"shell\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":%d," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\"{\\\"command\\\":" + "\\\"/usr/bin/jot -b x 140000; exit 1\\\"}\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":%d}\n\n", + i, i, i, i); + if (n < 0 || (size_t)n >= cap - off) + errx(1, "resultbound fixture too large"); + off += (size_t)n; + } + n = snprintf(sse + off, cap - off, + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":7}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n"); + if (n < 0 || (size_t)n >= cap - off) + errx(1, "resultbound fixture too large"); + off += (size_t)n; + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, sse, off); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + free(sse); +} + /* A web_fetch call is formatted with this stub's dynamic listen port; - * the same TLS origin serves /web without consuming an API scenario. */ + * the same TLS origin serves the path without consuming an API scenario. */ static void -send_webfetch(struct tls *conn, int port) +send_webfetch_path(struct tls *conn, int port, const char *path) { char sse[2048]; int n; @@ -138,7 +694,7 @@ send_webfetch(struct tls *conn, int port) "event: content_block_delta\n" "data: {\"type\":\"content_block_delta\",\"index\":0," "\"delta\":{\"type\":\"input_json_delta\"," - "\"partial_json\":\"{\\\"url\\\":\\\"https://127.0.0.1:%d/web" + "\"partial_json\":\"{\\\"url\\\":\\\"https://127.0.0.1:%d/%s" "\\\"}\"}}\n\n" "event: content_block_stop\n" "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" @@ -146,7 +702,7 @@ send_webfetch(struct tls *conn, int port) "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" "\"tool_use\"},\"usage\":{\"output_tokens\":12}}\n\n" "event: message_stop\n" - "data: {\"type\":\"message_stop\"}\n\n", port); + "data: {\"type\":\"message_stop\"}\n\n", port, path); if (n < 0 || (size_t)n >= sizeof(sse)) errx(1, "WEBFETCH fixture too large"); (void)tls_write_all(conn, body_ok, strlen(body_ok)); @@ -154,6 +710,59 @@ send_webfetch(struct tls *conn, int port) (void)tls_write_all(conn, "0\r\n\r\n", 5); } +static void +send_webfetch(struct tls *conn, int port) +{ + send_webfetch_path(conn, port, "web"); +} + +static void +send_wave_webfetch(struct tls *conn, int port, const char *req) +{ + if (strstr(req, "WAVE-PROMPT-A") != NULL) + send_webfetch_path(conn, port, "web-a"); + else if (strstr(req, "WAVE-PROMPT-B") != NULL) + send_webfetch_path(conn, port, "web-b"); + else + errx(1, "web child request has no wave prompt"); +} + +/* A subagent is networkless even when the persistent lead tool role is + * explicitly allowed to use the network. */ +static void +send_agentnet(struct tls *conn, int port) +{ + char sse[2048]; + int n; + + n = snprintf(sse, sizeof(sse), + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":17,\"output_tokens\":1}}}\n\n" + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":" + "\"toolu_net\",\"name\":\"shell\"}}\n\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\"{\\\"command\\\":\\\"if /usr/bin/nc " + "-z 127.0.0.1 %d; then echo SUBAGENT-NET-OPEN; else echo " + "SUBAGENT-NET-DENIED; exit 1; fi\\\"}\"}}\n\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"tool_use\"},\"usage\":{\"output_tokens\":9}}\n\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n", port); + if (n < 0 || (size_t)n >= sizeof(sse)) + errx(1, "AGENTNET fixture too large"); + (void)tls_write_all(conn, body_ok, strlen(body_ok)); + write_chunk(conn, sse, (size_t)n); + (void)tls_write_all(conn, "0\r\n\r\n", 5); +} + /* an assistant turn calling a shell sleep: the pause holds the turn at * its tool step so a pty script can queue and steer a message into the * next tool-result batch (behavior.md 2.1) */ @@ -332,6 +941,23 @@ tls_write_all(struct tls *tls, const char *p, size_t l } static void +send_pool503(struct tls *conn) +{ + char hdr[160]; + int n; + + n = snprintf(hdr, sizeof(hdr), + "HTTP/1.1 503 Service Unavailable\r\n" + "retry-after: 2\r\n" + "content-type: application/json\r\n" + "content-length: %zu\r\n\r\n", strlen(err503)); + if (n < 0 || (size_t)n >= sizeof(hdr)) + errx(1, "pool 503 fixture too large"); + (void)tls_write_all(conn, hdr, (size_t)n); + (void)tls_write_all(conn, err503, strlen(err503)); +} + +static void write_chunk(struct tls *tls, const char *s, size_t len) { char hdr[32]; @@ -427,18 +1053,75 @@ record_key(const char *req, const char *keyout) fclose(f); } +static int +pool_prompt_index(const char *req) +{ + char marker[32]; + int i; + + for (i = 0; i < 9; i++) { + (void)snprintf(marker, sizeof(marker), "POOL-PROMPT-%d", i); + if (strstr(req, marker) != NULL) + return (i); + } + return (-1); +} + +static int +pool_seen_count(unsigned int seen) +{ + int count = 0; + + while (seen != 0) { + count += seen & 1U; + seen >>= 1; + } + return (count); +} + +static void +record_pool_winner(const char *path, const char *req, unsigned int *seen, + const struct timespec *started) +{ + struct timespec now; + FILE *f; + long long elapsed; + unsigned int bit; + int before, fresh, index; + + index = pool_prompt_index(req); + if (index == -1) + errx(1, "POOLWIN request has no pool prompt"); + bit = 1U << index; + before = pool_seen_count(*seen); + fresh = (*seen & bit) == 0; + *seen |= bit; + clock_gettime(CLOCK_MONOTONIC, &now); + elapsed = (long long)(now.tv_sec - started->tv_sec) * 1000 + + (now.tv_nsec - started->tv_nsec) / 1000000; + if (path == NULL || (f = fopen(path, "w")) == NULL) + return; + fprintf(f, "%d %d %lld\n", fresh, before, elapsed); + fclose(f); +} + int main(int argc, char *argv[]) { struct tls_config *cfg; struct tls *srv, *conn; struct sockaddr_in sin; + struct timespec pool_started; socklen_t slen; const char *cert = NULL, *key = NULL, *keyout, *reqout, - *reqlog; - char req[65536]; + *reqlog, *overlapout, *poolout, + *weborderout, *futurepath; + char *req; + char web_first = '?', web_second = '?'; ssize_t reqlen; - int ch, i, s, c, listen_port; + unsigned int pool_seen = 0; + int ch, i, s, c, listen_port, pool_started_set = 0, + web_count = 0, web_serial = 1; while ((ch = getopt(argc, argv, "c:k:")) != -1) { switch (ch) { @@ -459,6 +1142,12 @@ main(int argc, char *argv[]) keyout = getenv("KEYOUT"); reqout = getenv("REQOUT"); reqlog = getenv("REQLOG"); + overlapout = getenv("OVERLAPOUT"); + poolout = getenv("POOLOUT"); + weborderout = getenv("WEBORDEROUT"); + futurepath = getenv("FUTUREPATH"); + if ((req = calloc(1, STUB_REQ_MAX)) == NULL) + errx(1, "request fixture allocation"); signal(SIGPIPE, SIG_IGN); @@ -496,16 +1185,59 @@ main(int argc, char *argv[]) close(c); continue; } - if ((reqlen = read_request(conn, req, sizeof(req))) > 0) { - if (strncmp(req, "GET /web HTTP/1.1", 17) == 0) { - static const char response[] = + if ((reqlen = read_request(conn, req, STUB_REQ_MAX)) > 0) { + const char *web_body = NULL; + char web_path = '\0'; + + if (strncmp(req, "GET /web HTTP/1.1", + sizeof("GET /web HTTP/1.1") - 1) == 0) { + web_body = "WEB-MESH-RESULT"; + } else if (strncmp(req, "GET /web-a HTTP/1.1", + sizeof("GET /web-a HTTP/1.1") - 1) == 0) { + web_body = "WEB-MESH-RESULT-A"; + web_path = 'a'; + } else if (strncmp(req, "GET /web-b HTTP/1.1", + sizeof("GET /web-b HTTP/1.1") - 1) == 0) { + web_body = "WEB-MESH-RESULT-B"; + web_path = 'b'; + } + if (web_body != NULL) { + char hdr[128]; + int hn; + + if (web_path != '\0') { + FILE *f; + + if (web_count == 0) { + struct pollfd pfd; + + memset(&pfd, 0, sizeof(pfd)); + pfd.fd = s; + pfd.events = POLLIN; + /* Both provider calls have returned. A second + * pending TCP connection here would mean the web + * broker ran the calls concurrently. */ + if (poll(&pfd, 1, 750) != 0) + web_serial = 0; + web_first = web_path; + } else if (web_count == 1) { + web_second = web_path; + } + web_count++; + if (web_count == 2 && weborderout != NULL && + (f = fopen(weborderout, "w")) != NULL) { + fprintf(f, "%d %d %c %c\n", web_serial, + web_count, web_first, web_second); + fclose(f); + } + } + hn = snprintf(hdr, sizeof(hdr), "HTTP/1.1 200 OK\r\n" "Content-Type: text/plain\r\n" - "Content-Length: 15\r\n\r\n" - "WEB-MESH-RESULT"; - - (void)tls_write_all(conn, response, - sizeof(response) - 1); + "Content-Length: %zu\r\n\r\n", + strlen(web_body)); + (void)tls_write_all(conn, hdr, (size_t)hn); + (void)tls_write_all(conn, web_body, strlen(web_body)); (void)tls_close(conn); tls_free(conn); close(c); @@ -516,8 +1248,10 @@ main(int argc, char *argv[]) /* the picker's live listing (behavior.md * 2.1): fixed ids, no scenario consumed */ static const char models[] = - "{\"data\":[{\"id\":\"stub-model-a\"}," - "{\"id\":\"stub-model-b\"}]}"; + "{\"data\":[{\"id\":\"stub-model-a\"," + "\"context_window\":111111},{\"id\":" + "\"stub-model-b\",\"context_window\":1000000001," + "\"context_length\":424242}]}"; char hdr[128]; int hn; @@ -561,6 +1295,305 @@ main(int argc, char *argv[]) write_chunk(conn, sse_tooluse, strlen(sse_tooluse)); (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTCALL") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentcall, + strlen(sse_agentcall)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "FUTURECALL") == 0) { + FILE *f; + + if (futurepath == NULL || + (f = fopen(futurepath, "w")) == NULL) + errx(1, "create future literal fixture"); + fputs("FUTURE-LITERAL-SECRET\n", f); + if (fclose(f) == EOF) + errx(1, "close future literal fixture"); + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_futurecall, + strlen(sse_futurecall)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "POOLCALL") == 0) { + send_poolcall(conn); + } else if (strcmp(argv[i], "AGENTREAD") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentread, + strlen(sse_agentread)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTPROTECT") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentprotect, + strlen(sse_agentprotect)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "FUTUREREAD") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_futureread, + strlen(sse_futureread)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTWRITE") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentwrite, + strlen(sse_agentwrite)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTNET") == 0) { + send_agentnet(conn, listen_port); + } else if (strcmp(argv[i], "AGENTROUND") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentread, + strlen(sse_agentread)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTHANG") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agenthang, + strlen(sse_agenthang)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTREPORT") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentreport, + strlen(sse_agentreport)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "FUTUREREPORT") == 0) { + send_text(conn, "FUTURE-LITERAL-PROTECTED-REPORT"); + } else if (strcmp(argv[i], "FUTURESYNTH") == 0) { + send_text(conn, "FUTURE-LITERAL-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "AGENTSYNTH") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentsynth, + strlen(sse_agentsynth)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTBAD") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentbad, + strlen(sse_agentbad)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "AGENTLABEL") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_agentlabel, + strlen(sse_agentlabel)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "WAVECALL") == 0) { + (void)tls_write_all(conn, body_ok, + strlen(body_ok)); + write_chunk(conn, sse_wavecall, + strlen(sse_wavecall)); + (void)tls_write_all(conn, "0\r\n\r\n", 5); + } else if (strcmp(argv[i], "WEBPROVA") == 0) { + struct pollfd pfd; + struct tls *conn2 = NULL; + char req2[65536]; + ssize_t len2; + int c2 = -1, got2 = 0; + + memset(&pfd, 0, sizeof(pfd)); + pfd.fd = s; + pfd.events = POLLIN; + if (i + 1 < argc && + strcmp(argv[i + 1], "WEBPROVB") == 0 && + poll(&pfd, 1, 3000) == 1 && + (c2 = accept(s, NULL, NULL)) != -1 && + tls_accept_socket(srv, &conn2, c2) != -1 && + (len2 = read_request(conn2, req2, + sizeof(req2))) > 0) { + FILE *f; + + got2 = 1; + record_key(req2, keyout); + record_req(req2, (size_t)len2, reqout); + record_req_log(req2, (size_t)len2, reqlog, + argv[i + 1]); + if (overlapout != NULL && + (f = fopen(overlapout, "w")) != NULL) { + fputs("two web children in flight\n", f); + fclose(f); + } + send_wave_webfetch(conn, listen_port, req); + send_wave_webfetch(conn2, listen_port, req2); + (void)tls_close(conn2); + tls_free(conn2); + close(c2); + i++; + } + if (!got2) { + if (conn2 != NULL) { + (void)tls_close(conn2); + tls_free(conn2); + } + if (c2 != -1) + close(c2); + send_wave_webfetch(conn, listen_port, req); + } + } else if (strcmp(argv[i], "WEBPROVB") == 0) { + send_wave_webfetch(conn, listen_port, req); + } else if (strcmp(argv[i], "WEBREPORT") == 0) { + if (strstr(req, "WEB-MESH-RESULT-A") != NULL) + send_text(conn, "WEB-REPORT-A"); + else if (strstr(req, "WEB-MESH-RESULT-B") != NULL) + send_text(conn, "WEB-REPORT-B"); + else + errx(1, "web report request has no brokered result"); + } else if (strcmp(argv[i], "WEBSYNTH") == 0) { + send_text(conn, "WEB-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "WAVEA") == 0) { + send_text(conn, "WAVE-REPORT-A"); + } else if (strcmp(argv[i], "WAVEB") == 0) { + send_text(conn, "WAVE-REPORT-B"); + } else if (strcmp(argv[i], "WAVESYNTH") == 0) { + send_text(conn, "WAVE-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "OVERLAPA") == 0) { + struct pollfd pfd; + struct tls *conn2 = NULL; + char req2[65536]; + ssize_t len2; + int c2; + const char *mark1, *mark2; + + mark1 = strstr(req, "WAVE-PROMPT-A") != NULL ? + "OVERLAP-REPORT-A" : "OVERLAP-REPORT-B"; + memset(&pfd, 0, sizeof(pfd)); + pfd.fd = s; + pfd.events = POLLIN; + if (i + 1 < argc && + strcmp(argv[i + 1], "OVERLAPB") == 0 && + poll(&pfd, 1, 3000) == 1 && + (c2 = accept(s, NULL, NULL)) != -1 && + tls_accept_socket(srv, &conn2, c2) != -1 && + (len2 = read_request(conn2, req2, + sizeof(req2))) > 0) { + FILE *f; + + record_key(req2, keyout); + record_req(req2, (size_t)len2, reqout); + record_req_log(req2, (size_t)len2, reqlog, + argv[i + 1]); + if (overlapout != NULL && + (f = fopen(overlapout, "w")) != NULL) { + fputs("two requests in flight\n", f); + fclose(f); + } + mark2 = strstr(req2, "WAVE-PROMPT-A") != NULL ? + "OVERLAP-REPORT-A" : "OVERLAP-REPORT-B"; + /* Complete the later request first: correlation, + * not arrival order, must select the child. */ + send_text(conn2, mark2); + (void)tls_close(conn2); + tls_free(conn2); + close(c2); + i++; + } + send_text(conn, mark1); + } else if (strcmp(argv[i], "OVERLAPB") == 0) { + /* Reached only when no second request overlapped. */ + send_text(conn, + strstr(req, "WAVE-PROMPT-A") != NULL ? + "OVERLAP-REPORT-A" : "OVERLAP-REPORT-B"); + } else if (strcmp(argv[i], "OVERLAPSYNTH") == 0) { + send_text(conn, "OVERLAP-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "POOL503") == 0) { + struct pollfd pfd; + struct tls *held[7]; + char req2[65536]; + ssize_t len2; + unsigned int bit; + int c2, heldfd[7], index, j; + + index = pool_prompt_index(req); + if (index == -1) + errx(1, "POOL503 request has no pool prompt"); + pool_seen = 1U << index; + memset(held, 0, sizeof(held)); + memset(&pfd, 0, sizeof(pfd)); + pfd.fd = s; + pfd.events = POLLIN; + for (j = 0; j < 7; j++) { + if (i + j + 1 >= argc || + strcmp(argv[i + j + 1], "POOL503") != 0 || + poll(&pfd, 1, 3000) != 1 || + (c2 = accept(s, NULL, NULL)) == -1 || + tls_accept_socket(srv, &held[j], c2) == -1 || + (len2 = read_request(held[j], req2, + sizeof(req2))) <= 0) + errx(1, "pool did not fill eight network slots"); + heldfd[j] = c2; + record_key(req2, keyout); + record_req(req2, (size_t)len2, reqout); + record_req_log(req2, (size_t)len2, reqlog, + argv[i + j + 1]); + index = pool_prompt_index(req2); + if (index == -1) + errx(1, "POOL503 request has no pool prompt"); + bit = 1U << index; + if (pool_seen & bit) + errx(1, "pool filled with a duplicate request"); + pool_seen |= bit; + } + if (pool_seen_count(pool_seen) != 8) + errx(1, "pool did not hold eight distinct requests"); + /* All live slots are occupied. Let the ninth worker's END + * reach fugu-api before releasing the sockets together. */ + usleep(100000); + clock_gettime(CLOCK_MONOTONIC, &pool_started); + pool_started_set = 1; + send_pool503(conn); + for (j = 0; j < 7; j++) { + send_pool503(held[j]); + (void)tls_close(held[j]); + tls_free(held[j]); + close(heldfd[j]); + } + i += 7; + } else if (strcmp(argv[i], "POOLWIN") == 0) { + if (!pool_started_set) + errx(1, "POOLWIN preceded POOL503"); + record_pool_winner(poolout, req, &pool_seen, + &pool_started); + send_text(conn, "POOL-FAST-REPORT"); + } else if (strcmp(argv[i], "POOLBACK") == 0) { + int index = pool_prompt_index(req); + + if (index == -1 || !(pool_seen & (1U << index))) + errx(1, "POOLBACK request was not an admitted retry"); + send_text(conn, "POOL-BACKOFF-REPORT"); + } else if (strcmp(argv[i], "POOLSYNTH") == 0) { + send_text(conn, "POOL-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "LIMITTEXT") == 0) { + send_limit(conn, 0); + } else if (strcmp(argv[i], "LIMITCALL") == 0) { + send_limit(conn, 1); + } else if (strcmp(argv[i], "LIMITCONCLUDE") == 0) { + send_text(conn, "LIMIT-CONCLUSION-TEXT"); + } else if (strcmp(argv[i], "LIMITSYNTH") == 0) { + send_text(conn, "LIMIT-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "RETAINLIMIT") == 0) { + send_retain(conn, 0); + } else if (strcmp(argv[i], "RETAINTOOL") == 0) { + send_retain(conn, 1); + } else if (strcmp(argv[i], "RETAINCONCLUDE") == 0) { + send_text(conn, + "RETAIN-FINAL-CONCLUSION-DIAGNOSTIC"); + } else if (strcmp(argv[i], "RETAINSYNTH") == 0) { + send_text(conn, "RETAIN-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "RETAINERRSYNTH") == 0) { + send_text(conn, "RETAIN-ERROR-SYNTHESIS-MARKER"); + } else if (strcmp(argv[i], "ARGBOUND") == 0) { + send_argbound(conn); + } else if (strcmp(argv[i], "RESULTBOUND") == 0) { + send_resultbound(conn); + } else if (strcmp(argv[i], "AGENTCONCLUDE") == 0) { + send_text(conn, "BOUND-REPORT-MARKER"); + } else if (strcmp(argv[i], "BOUNDSYNTH") == 0) { + send_text(conn, "BOUND-SYNTHESIS-MARKER"); } else if (strcmp(argv[i], "WEBFETCH") == 0) { send_webfetch(conn, listen_port); } else if (strcmp(argv[i], "EDITDIFF") == 0) { @@ -665,5 +1698,6 @@ main(int argc, char *argv[]) tls_free(conn); close(c); } + free(req); return (0); } blob - /dev/null blob + b27e0492adfb7c899e5452cb5551a1000c439953 (mode 755) --- /dev/null +++ scripts/live-check @@ -0,0 +1,789 @@ +#!/bin/sh +# +# Copyright (c) 2026 Isaac +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +# Credentialed and privileged M10 release checks. This is deliberately not +# part of `make check`: it spends provider/search quota, reaches the public +# network, and (in privileged mode) installs a package on a clean host. + +set -u +umask 077 +# OpenBSD sh supports the base-system core-size limit used here. +# shellcheck disable=SC3045 +ulimit -c 0 2>/dev/null || : + +prog=${0##*/} +root=$(cd "$(dirname "$0")/.." && pwd) +checks=0 +fails=0 +skips=0 +tmp= +restore_conf=0 +conf_backup= +installed_conf= +conf_restore_tmp= + +pass() +{ + checks=$((checks + 1)) + printf 'ok %d - %s\n' "$checks" "$1" +} + +fail() +{ + checks=$((checks + 1)) + fails=$((fails + 1)) + printf 'FAIL %d - %s\n' "$checks" "$1" +} + +skip() +{ + checks=$((checks + 1)) + skips=$((skips + 1)) + printf 'SKIP %d - %s\n' "$checks" "$1" +} + +ok() +{ + if [ "$2" -eq 0 ]; then + pass "$1" + else + fail "$1" + fi +} + +ok_zero() +{ + if [ "$2" -eq 0 ]; then + pass "$1" + else + fail "$1" + fi +} + +die() +{ + printf '%s: %s\n' "$prog" "$*" >&2 + exit 2 +} + +finish() +{ + printf '%d checks, %d failures, %d skipped\n' \ + "$checks" "$fails" "$skips" + [ "$fails" -eq 0 ] +} + +restore_admin_conf() +{ + if [ "$restore_conf" -eq 1 ] && [ -n "$conf_backup" ] && + [ -n "$installed_conf" ]; then + conf_restore_tmp=${installed_conf}.live-restore.$$ + if install -o root -g _fugu -m 0640 "$conf_backup" \ + "$conf_restore_tmp" >/dev/null 2>&1 && + mv -f "$conf_restore_tmp" "$installed_conf"; then + restore_conf=0 + conf_restore_tmp= + else + printf '%s: WARNING: could not restore %s\n' \ + "$prog" "$installed_conf" >&2 + fi + fi +} + +cleanup() +{ + restore_admin_conf + if [ "$restore_conf" -eq 1 ]; then + printf '%s: CRITICAL: secure configuration backup retained at %s\n' \ + "$prog" "$conf_backup" >&2 + tmp= + trap - EXIT + exit 1 + fi + if [ -n "$conf_restore_tmp" ]; then + rm -f "$conf_restore_tmp" + fi + if [ -n "$tmp" ] && [ -d "$tmp" ]; then + rm -rf "$tmp" + fi +} + +trap cleanup EXIT +trap 'exit 1' HUP INT TERM + +usage() +{ + cat </dev/null +} + +credentialed_run() +{ + ( + cd "$live_workdir" || exit 1 + mode=$(stat -f %Mp%Lp "$live_bin" 2>/dev/null || printf unknown) + if [ "$mode" = 2555 ]; then + # Installed setgid execution must ignore every development endpoint + # override, including those consumed only after worker exec. + exec env HOME="$live_home" FUGU_CONF=/nonexistent \ + FUGU_LIBEXEC=/nonexistent FUGU_CA_FILE=/nonexistent \ + FUGU_KAGI_HOST=127.0.0.1 FUGU_KAGI_PORT=1 \ + "$live_bin" "$@" /dev/null 2>&1 + return $? + fi + if command -v "$live_doas" >/dev/null 2>&1; then + "$live_doas" env LC_ALL=C awk "$program" "$key" \ + >/dev/null 2>&1 + return $? + fi + return 1 +} + +# Set tool_result_line and prove one call has one correlated result and that +# no unrelated tool call/result can satisfy the assertion. +one_tool_result() +{ + file=$1 + name=$2 + is_error=$3 + all_calls=$(grep -c '"type":"tool_call"' "$file" 2>/dev/null || :) + all_results=$(grep -c '"type":"tool_result"' "$file" 2>/dev/null || :) + ids=$(sed -n 's/.*"type":"tool_call","id":"\([^"]*\)","name":"'"$name"'".*/\1/p' \ + "$file") + [ "$all_calls" -eq 1 ] && [ "$all_results" -eq 1 ] || return 1 + [ "$(printf '%s\n' "$ids" | sed '/^$/d' | wc -l)" -eq 1 ] || return 1 + tool_id=$ids + tool_result_line=$(grep -F \ + "\"type\":\"tool_result\",\"id\":\"$tool_id\"" \ + "$file" 2>/dev/null) + [ "$(printf '%s\n' "$tool_result_line" | sed '/^$/d' | wc -l)" -eq 1 ] || + return 1 + printf '%s\n' "$tool_result_line" | + grep -F -q "\"is_error\":$is_error" +} + +# Return 0 when absent, 1 when leaked, and 2 when the scan could not run. +secret_absent() +{ + key=$1 + target=$2 + if [ -r "$key" ]; then + grep -R -F -q -f "$key" "$target" >/dev/null 2>&1 + rc=$? + else + command -v "$live_doas" >/dev/null 2>&1 || return 2 + "$live_doas" grep -R -F -q -f "$key" "$target" \ + >/dev/null 2>&1 + rc=$? + fi + case $rc in + 0) return 1 ;; + 1) return 0 ;; + *) return 2 ;; + esac +} + +syslog_has() +{ + marker=$1 + if [ -r "$live_syslog" ]; then + grep -F -q "$marker" "$live_syslog" 2>/dev/null + return $? + fi + command -v "$live_doas" >/dev/null 2>&1 || return 2 + "$live_doas" grep -F -q "$marker" "$live_syslog" >/dev/null 2>&1 +} + +syslog_secret_absent() +{ + key=$1 + start=$2 + end=$3 + # The program reads the one-line key first, then returns only an exit + # status for the delimited log slice. No privileged log byte is emitted. + # shellcheck disable=SC2016 + program='NR == FNR { if (FNR == 1) key = $0; next } + index($0, start) { copying = 1; saw_start = 1 } + copying && index($0, key) { leaked = 1 } + copying && index($0, end) { saw_end = 1; exit } + END { + if (!saw_start || !saw_end) exit 2 + if (leaked) exit 1 + exit 0 + }' + if [ -r "$key" ] && [ -r "$live_syslog" ]; then + LC_ALL=C awk -v start="$start" -v end="$end" "$program" \ + "$key" "$live_syslog" >/dev/null 2>&1 + return $? + fi + command -v "$live_doas" >/dev/null 2>&1 || return 2 + "$live_doas" env LC_ALL=C awk -v start="$start" -v end="$end" \ + "$program" "$key" "$live_syslog" >/dev/null 2>&1 +} + +wait_syslog_marker() +{ + marker=$1 + i=0 + while [ "$i" -lt 20 ]; do + syslog_has "$marker" + rc=$? + [ "$rc" -eq 0 ] && return 0 + [ "$rc" -eq 2 ] && return 2 + sleep 1 + i=$((i + 1)) + done + return 1 +} + +credentialed() +{ + live_bin=${FUGU_LIVE_BIN:-/usr/local/bin/fugu} + live_conf=${FUGU_LIVE_CONF:-/etc/fugu.conf} + live_libexec=${FUGU_LIVE_LIBEXEC:-} + live_workdir=${FUGU_LIVE_WORKDIR:-$root} + live_syslog=${FUGU_LIVE_SYSLOG:-/var/log/messages} + live_doas=${FUGU_LIVE_DOAS:-doas} + expect_model=${FUGU_LIVE_EXPECT_MODEL:-claude-sonnet-4-6} + expect_provider=${FUGU_LIVE_EXPECT_PROVIDER:-anthropic} + secret_files=${FUGU_LIVE_SECRET_FILES:-} + expired_conf=${FUGU_LIVE_EXPIRED_PROVIDER_CONF:-} + expired_bin=${FUGU_LIVE_EXPIRED_PROVIDER_BIN:-} + + [ "$(uname -s)" = OpenBSD ] && [ "$(uname -r)" = 7.9 ] || + die "credentialed checks require OpenBSD 7.9" + [ "$(id -u)" -ne 0 ] || die "run credentialed checks as an ordinary user" + [ -x "$live_bin" ] || die "$live_bin is not executable" + [ -d "$live_workdir" ] || die "$live_workdir is not a directory" + [ -n "$secret_files" ] || die "FUGU_LIVE_SECRET_FILES is required" + [ -e "$live_syslog" ] || die "$live_syslog does not exist" + for key in $secret_files; do + secret_file_ok "$key" || + die "$key must contain exactly one nonempty printable key line" + done + if [ -n "$expired_conf" ]; then + [ -f "$expired_conf" ] || die "$expired_conf is not a file" + [ -n "$expired_bin" ] && [ -x "$expired_bin" ] || + die "FUGU_LIVE_EXPIRED_PROVIDER_BIN must name a non-setgid binary" + expired_mode=$(stat -f %Mp%Lp "$expired_bin" 2>/dev/null) + case $expired_mode in + [2-7][0-7][0-7][0-7]) + die "expired provider check cannot use a set-ID binary" + ;; + esac + fi + + tmp=$(mktemp -d /tmp/fugu-live.XXXXXXXXXX) || exit 1 + chmod 0700 "$tmp" + artifacts=$tmp/artifacts + live_home=$tmp/home + project=$tmp/project + mkdir -p "$artifacts" "$live_home" "$project" + chmod 0700 "$artifacts" "$live_home" "$project" + log_nonce=${tmp##*/} + log_begin="$log_nonce-begin" + log_end="$log_nonce-end" + syslog_has "$log_begin" + [ "$?" -eq 1 ] || die "syslog begin marker is not unique" + syslog_has "$log_end" + [ "$?" -eq 1 ] || die "syslog end marker is not unique" + logger -t fugu-live "$log_begin" || die "cannot write syslog marker" + wait_syslog_marker "$log_begin" || die "syslog begin marker was not stored" + + printf '%s\n' "LIVE-CHILD-LOCAL-A-$$" >"$project/alpha.txt" + printf '%s\n' "LIVE-CHILD-LOCAL-B-$$" >"$project/beta.txt" + local_a="LIVE-CHILD-LOCAL-A-$$" + local_b="LIVE-CHILD-LOCAL-B-$$" + report_a="LIVE-AGENT-A-REPORT-$$" + report_b="LIVE-AGENT-B-REPORT-$$" + + printf '%s\n' \ + "# credentialed checks make paid provider and Kagi requests" + + credentialed_run -n >"$artifacts/config.out" \ + 2>"$artifacts/config.err" + rc=$? + ok_zero "effective configuration parses" "$rc" + line_is "$artifacts/config.out" "model \"$expect_model\"" + ok "effective model is $expect_model" $? + line_is "$artifacts/config.out" "provider \"$expect_provider\"" + ok "effective provider is $expect_provider" $? + line_is "$artifacts/config.out" "api_key " + ok "default provider key is configured and redacted" $? + line_is "$artifacts/config.out" "kagi_token " + ok "Kagi key is configured and redacted" $? + line_is "$artifacts/config.out" "web_search yes" + ok "web tools are enabled" $? + awk '$1 == "max_subagents" && $2 + 0 >= 2 { found = 1 } + END { exit !found }' "$artifacts/config.out" + ok "at least two subagent slots are configured" $? + subroute=$(sed -n 's/^subagent_provider "\(.*\)"$/\1/p' \ + "$artifacts/config.out") + [ -n "$subroute" ] && [ "$subroute" != "@default" ] && + line_is "$artifacts/config.out" "provider \"$subroute\" {" + ok "subagents route through a named provider" $? + grep -q '^subagent_model "' "$artifacts/config.out" 2>/dev/null + ok "a subagent model override is configured" $? + + credentialed_run -p 'Reply with exactly: live provider ok' \ + >"$artifacts/default.out" 2>"$artifacts/default.err" + rc=$? + ok_zero "default-provider turn exits zero" "$rc" + line_is "$artifacts/default.out" "live provider ok" + ok "default-provider turn returns the exact marker" $? + + live_workdir=$project + subprompt="Call the agent tool exactly twice in the same response and no other tool. Label the calls live-a and live-b. live-a must read alpha.txt and then return exactly $report_a without quoting the file. live-b must read beta.txt and then return exactly $report_b without quoting the file. After both reports arrive, reply exactly: live subagents ok" + credentialed_run -p "$subprompt" -o ndjson \ + >"$artifacts/subagent.out" 2>"$artifacts/subagent.err" + rc=$? + ok_zero "routed-subagent turn exits zero" "$rc" + n=$(grep -c '"type":"tool_call".*"name":"agent"' \ + "$artifacts/subagent.out" 2>/dev/null || :) + all=$(grep -c '"type":"tool_call"' "$artifacts/subagent.out" \ + 2>/dev/null || :) + if [ "$n" -eq 2 ] && [ "$all" -eq 2 ]; then + rc=0 + else + rc=1 + fi + ok "lead makes exactly two agent calls" "$rc" + grep -F -q '\"label\":\"live-a\"' "$artifacts/subagent.out" && + grep -F -q '\"label\":\"live-b\"' "$artifacts/subagent.out" + ok "both requested agent labels reach the tool calls" $? + grep -q 'agent\[live-a\]: started' "$artifacts/subagent.err" && + grep -q 'agent\[live-b\]: started' "$artifacts/subagent.err" && + grep -q 'agent\[live-a\]: completed' "$artifacts/subagent.err" && + grep -q 'agent\[live-b\]: completed' "$artifacts/subagent.err" + ok "both routed agents expose start and completion progress" $? + grep -q 'agent\[live-a\]: tool read' "$artifacts/subagent.err" && + grep -q 'agent\[live-b\]: tool read' "$artifacts/subagent.err" + ok "both routed agents execute their read tool" $? + a_start=$(grep -n 'agent\[live-a\]: started' "$artifacts/subagent.err" | + head -1 | cut -d: -f1) + b_start=$(grep -n 'agent\[live-b\]: started' "$artifacts/subagent.err" | + head -1 | cut -d: -f1) + first_done=$(grep -n 'agent\[live-[ab]\]: completed' \ + "$artifacts/subagent.err" | head -1 | cut -d: -f1) + if [ -n "$a_start" ] && [ -n "$b_start" ] && + [ -n "$first_done" ] && [ "$a_start" -lt "$first_done" ] && + [ "$b_start" -lt "$first_done" ]; then + rc=0 + else + rc=1 + fi + ok "both agents start before either one completes" "$rc" + n=$(grep -c '"type":"tool_result".*"is_error":false' \ + "$artifacts/subagent.out" 2>/dev/null || :) + all=$(grep -c '"type":"tool_result"' "$artifacts/subagent.out" \ + 2>/dev/null || :) + [ "$n" -eq 2 ] && [ "$all" -eq 2 ] && + grep -q "$report_a" "$artifacts/subagent.out" && + grep -q "$report_b" "$artifacts/subagent.out" + ok "both bounded agent reports return successfully" $? + journals=$live_home/.fugu/sessions + grep -R -q '"name":"agent"' "$journals" 2>/dev/null && + grep -R -q "$report_a" "$journals" 2>/dev/null && + grep -R -q "$report_b" "$journals" 2>/dev/null + ok "lead agent calls and final reports are journaled" $? + if grep -R -E -q "$local_a|$local_b|ephemeral read-only subagent|\"name\":\"read\"" \ + "$journals" 2>/dev/null; then + rc=1 + else + rc=0 + fi + ok "child system text and local read results stay out of the journal" "$rc" + + live_workdir=$root + credentialed_run -p 'Use web_search exactly once for OpenBSD 7.9 release, then summarize the first result.' -o ndjson \ + >"$artifacts/search.out" 2>"$artifacts/search.err" + rc=$? + ok_zero "live Kagi search turn exits zero" "$rc" + one_tool_result "$artifacts/search.out" web_search false + ok "Kagi search returns a non-error broker result" $? + + credentialed_run -p 'Use web_fetch exactly once on https://example.com/ and report its title.' -o ndjson \ + >"$artifacts/fetch.out" 2>"$artifacts/fetch.err" + rc=$? + ok_zero "live public fetch turn exits zero" "$rc" + one_tool_result "$artifacts/fetch.out" web_fetch false && + printf '%s\n' "$tool_result_line" | grep -F -q 'Example Domain' + ok "web_fetch returns the public Example Domain text" $? + + credentialed_run -p 'Use web_fetch exactly once on https://expired.badssl.com/. Report the exact TLS error and do not use another tool.' -o ndjson \ + >"$artifacts/expired.out" 2>"$artifacts/expired.err" + rc=$? + ok_zero "expired-certificate tool turn itself completes" "$rc" + one_tool_result "$artifacts/expired.out" web_fetch true && + printf '%s\n' "$tool_result_line" | + grep -Eqi 'certificate[^\"]*(expired|not[ -]?after)' + ok "expired.badssl.com surfaces a real certificate error" $? + + if [ -n "$expired_conf" ]; then + old_conf=$live_conf + old_bin=$live_bin + live_conf=$expired_conf + live_bin=$expired_bin + credentialed_run -p 'provider certificate check' -o ndjson \ + >"$artifacts/provider-expired.out" \ + 2>"$artifacts/provider-expired.err" + rc=$? + [ "$rc" -eq 1 ] && + grep -Eqi 'certificate.*(expired|not[ -]?after)' \ + "$artifacts/provider-expired.out" \ + "$artifacts/provider-expired.err" + ok "expired provider endpoint fails with a certificate error" $? + live_conf=$old_conf + live_bin=$old_bin + else + skip "expired provider endpoint (no test endpoint configured)" + fi + + logger -t fugu-live "$log_end" || die "cannot write syslog marker" + wait_syslog_marker "$log_end" || die "syslog end marker was not stored" + syslog_has "$log_begin" && syslog_has "$log_end" + ok "syslog evidence is delimited by both run markers" $? + + for key in $secret_files; do + secret_absent "$key" "$artifacts" + rc=$? + ok "$(basename "$key") is absent from terminal output" "$rc" + secret_absent "$key" "$journals" + rc=$? + ok "$(basename "$key") is absent from session journals" "$rc" + syslog_secret_absent "$key" "$log_begin" "$log_end" + rc=$? + ok "$(basename "$key") is absent from syslog" "$rc" + done + + finish +} + +get_group() +{ + getent group "$1" 2>/dev/null +} + +run_as_user() +{ + chroot -u "$live_user" / /usr/bin/env HOME="$live_user_home" "$@" +} + +privileged() +{ + package=${FUGU_LIVE_PACKAGE:-} + conf_source=${FUGU_LIVE_CONF_SOURCE:-} + secret_files=${FUGU_LIVE_SECRET_FILES:-} + live_user=${FUGU_LIVE_USER:-} + confirm=${FUGU_LIVE_CONFIRM:-} + prefix=${FUGU_LIVE_PREFIX:-/usr/local} + sysconfdir=${FUGU_LIVE_SYSCONFDIR:-/etc} + expect_model=${FUGU_LIVE_EXPECT_MODEL:-claude-sonnet-4-6} + expect_provider=${FUGU_LIVE_EXPECT_PROVIDER:-anthropic} + live_doas=doas + bin=$prefix/bin/fugu + installed_conf=$sysconfdir/fugu.conf + sample=$sysconfdir/fugu.conf.sample + + [ "$(uname -s)" = OpenBSD ] && [ "$(uname -r)" = 7.9 ] || + die "privileged checks require OpenBSD 7.9" + [ "$(id -u)" -eq 0 ] || die "privileged checks must run as root" + [ -n "$package" ] || die "FUGU_LIVE_PACKAGE is required" + [ -f "$package" ] || die "$package is not a package file" + [ -n "$conf_source" ] || die "FUGU_LIVE_CONF_SOURCE is required" + [ -f "$conf_source" ] || die "$conf_source is not a file" + [ "$(stat -f %Su "$conf_source" 2>/dev/null)" = root ] || + die "$conf_source must be root-owned" + case $(stat -f %Lp "$conf_source" 2>/dev/null) in + 400|600) ;; + *) die "$conf_source must be mode 0400 or 0600" ;; + esac + if grep -Eq '^[[:space:]]*(api_key|anthropic_key|kagi_token)[[:space:]]+' \ + "$conf_source"; then + die "$conf_source contains an inline credential; use key files" + fi + if grep -Eq '^[[:space:]]*include[[:space:]]+' "$conf_source"; then + die "$conf_source must be self-contained for the live audit" + fi + if grep -Eq '^[[:space:]]*([[:alnum:]_]+[[:space:]]*=|\$)' \ + "$conf_source"; then + die "$conf_source must be macro-free for the live audit" + fi + [ -n "$secret_files" ] || die "FUGU_LIVE_SECRET_FILES is required" + for key in $secret_files; do + [ -f "$key" ] || die "$key is not a key file" + [ "$(stat -f %Su "$key" 2>/dev/null)" = root ] || + die "$key must be root-owned" + case $(stat -f %Lp "$key" 2>/dev/null) in + 400|600) ;; + *) die "$key must be mode 0400 or 0600 before installation" ;; + esac + secret_file_ok "$key" || + die "$key must contain exactly one nonempty printable key line" + done + [ -n "$live_user" ] || die "FUGU_LIVE_USER is required" + id "$live_user" >/dev/null 2>&1 || die "no such user: $live_user" + [ "$live_user" != root ] || die "FUGU_LIVE_USER must not be root" + command -v chroot >/dev/null 2>&1 || die "chroot is required" + if get_group _fugu >/dev/null 2>&1; then + die "_fugu already exists; use a clean package-test host" + fi + if getent group 992 >/dev/null 2>&1; then + die "GID 992 is already registered; use a clean package-test host" + fi + if pkg_info -e 'fugu-*' >/dev/null 2>&1; then + die "a fugu package is already registered; use a clean package-test host" + fi + [ ! -e "$bin" ] || die "$bin already exists; use a clean package-test host" + [ ! -e "$prefix/libexec/fugu" ] || + die "$prefix/libexec/fugu already exists; use a clean package-test host" + [ ! -e "$installed_conf" ] || + die "$installed_conf already exists; use a clean package-test host" + [ ! -e "$sample" ] || + die "$sample already exists; use a clean package-test host" + + live_user_home=$(getent passwd "$live_user" | awk -F: 'NR == 1 {print $6}') + [ -n "$live_user_home" ] || die "cannot resolve home for $live_user" + + tmp=$(mktemp -d /tmp/fugu-live-root.XXXXXXXXXX) || exit 1 + chmod 0700 "$tmp" + package_copy=$tmp/package.tgz + cp "$package" "$package_copy" || die "cannot snapshot package" + package_sha=$(sha256 -q "$package_copy") || die "cannot hash package" + want_confirm="$(hostname):$package_sha" + [ "$confirm" = "$want_confirm" ] || + die "set FUGU_LIVE_CONFIRM to $want_confirm" + + pkg_add "$package_copy" >"$tmp/pkg.out" 2>"$tmp/pkg.err" + rc=$? + ok "pkg_add installs the generated package" "$rc" + [ "$rc" -eq 0 ] || { finish; return $?; } + + group_line=$(get_group _fugu) + group_gid=$(printf '%s\n' "$group_line" | awk -F: 'NR == 1 {print $3}') + if [ "$group_gid" = 992 ]; then + rc=0 + else + rc=1 + fi + ok "package creates group _fugu with GID 992" "$rc" + pkg_info -e 'fugu-*' >/dev/null 2>&1 + ok "installed fugu package is registered" $? + [ "$(stat -f '%Su:%Sg:%Mp%Lp' "$bin" 2>/dev/null)" = \ + "root:_fugu:2555" ] + ok "installed front binary is root:_fugu mode 2555" $? + + workers='fugu-api fugu-editor fugu-tool fugu-tty fugu-web' + rc=0 + for worker in $workers; do + path=$prefix/libexec/fugu/$worker + [ -x "$path" ] && + [ "$(stat -f '%Su:%Sg:%Lp' "$path" 2>/dev/null)" = \ + "root:wheel:555" ] || + rc=1 + done + ok "all five workers are root:wheel mode 0555" "$rc" + [ -f "$prefix/man/man1/fugu.1" ] && + [ -f "$prefix/man/man5/fugu.conf.5" ] && + [ "$(stat -f '%Su:%Sg:%Lp' "$prefix/man/man1/fugu.1")" = \ + "root:wheel:444" ] && + [ "$(stat -f '%Su:%Sg:%Lp' "$prefix/man/man5/fugu.conf.5")" = \ + "root:wheel:444" ] + ok "both manuals are root:wheel mode 0444" $? + [ -f "$sample" ] && + [ "$(stat -f '%Su:%Sg:%Lp' "$sample" 2>/dev/null)" = \ + "root:_fugu:640" ] + ok "configuration sample is root:_fugu mode 0640" $? + + if id -Gn "$live_user" | tr ' ' '\n' | grep -qx _fugu; then + rc=1 + else + rc=0 + fi + ok "$live_user is not a member of _fugu" "$rc" + + rc=0 + for key in $secret_files; do + chown root:_fugu "$key" && chmod 0640 "$key" || rc=1 + done + ok "all configured key files are root:_fugu mode 0640" "$rc" + [ "$rc" -eq 0 ] || { finish; return $?; } + rc=0 + for key in $secret_files; do + chroot -u "$live_user" / /bin/test ! -r "$key" || rc=1 + done + ok "$live_user cannot read configured key files directly" "$rc" + + install -o root -g _fugu -m 0640 "$conf_source" "$installed_conf" + rc=$? + ok "operator configuration installs root:_fugu mode 0640" "$rc" + [ "$rc" -eq 0 ] || { finish; return $?; } + chroot -u "$live_user" / /bin/test ! -r "$installed_conf" + ok "$live_user cannot read the admin configuration directly" $? + + run_as_user "$bin" -n >"$tmp/user-config.out" \ + 2>"$tmp/user-config.err" + rc=$? + ok_zero "ordinary user reads the protected configuration through setgid" \ + "$rc" + line_is "$tmp/user-config.out" "model \"$expect_model\"" && + line_is "$tmp/user-config.out" "provider \"$expect_provider\"" && + line_is "$tmp/user-config.out" "api_key " + ok "ordinary-user dry run reports admin values with a redacted key" $? + + user_tmp=$tmp/user + mkdir "$user_tmp" + user_group=$(id -gn "$live_user") + chown "$live_user:$user_group" "$user_tmp" + chmod 0700 "$user_tmp" + chmod 0711 "$tmp" + alt=$user_tmp/override.conf + printf '%s\n' 'model "override-must-not-win"' 'provider "openai"' \ + >"$alt" + chown "$live_user:$user_group" "$alt" + chmod 0600 "$alt" + chroot -u "$live_user" / /bin/test -r "$alt" + ok "user-owned override fixture is directly readable" $? + run_as_user env FUGU_CONF="$alt" "$bin" -n \ + >"$tmp/override.out" 2>"$tmp/override.err" + rc=$? + [ "$rc" -eq 0 ] && + line_is "$tmp/override.out" "model \"$expect_model\"" && + ! grep -q 'override-must-not-win' "$tmp/override.out" + ok "setgid execution ignores a user-owned FUGU_CONF override" $? + + conf_backup=$tmp/fugu.conf.backup + cp -p "$installed_conf" "$conf_backup" || + die "cannot create secure configuration backup" + cmp -s "$installed_conf" "$conf_backup" && + [ "$(stat -f '%Su:%Sg:%Lp' "$conf_backup" 2>/dev/null)" = \ + "root:_fugu:640" ] || die "secure configuration backup is invalid" + restore_conf=1 + decoy=$tmp/fugu.conf.decoy + printf 'model "%s"\nprovider "%s"\n' \ + "$expect_model" "$expect_provider" >"$decoy" + chown root:_fugu "$decoy" + chmod 0644 "$decoy" + mv -f "$decoy" "$installed_conf" + run_as_user "$bin" -n >"$tmp/world.out" 2>"$tmp/world.err" + rc=$? + [ "$rc" -ne 0 ] && [ ! -s "$tmp/world.out" ] && + grep -Eqi 'group writable|world read|insecure' \ + "$tmp/world.err" + ok "world-readable admin configuration fails before startup" $? + restore_admin_conf + + printf 'model "%s"\nprovider "%s"\n' \ + "$expect_model" "$expect_provider" >"$decoy" + chown root:_fugu "$decoy" + chmod 0660 "$decoy" + restore_conf=1 + mv -f "$decoy" "$installed_conf" + run_as_user "$bin" -n >"$tmp/group.out" 2>"$tmp/group.err" + rc=$? + [ "$rc" -ne 0 ] && [ ! -s "$tmp/group.out" ] && + grep -Eqi 'group writable|world read|insecure' \ + "$tmp/group.err" + ok "group-writable admin configuration fails before startup" $? + restore_admin_conf + + finish +} + +case ${1:-help} in +credentialed) + credentialed + ;; +privileged) + privileged + ;; +help|-h|--help) + usage + ;; +*) + usage >&2 + exit 2 + ;; +esac blob - 0f835de388fd1eef0d80236514b25e7e673ebf31 blob + 3d53d396c049d07f6e55927ae0297e7c0462839f --- src/common/buf.c +++ src/common/buf.c @@ -54,23 +54,34 @@ buf_reset(struct buf *b) b->len = 0; } -static void -buf_reserve(struct buf *b, size_t n) +size_t +buf_capacity_after(const struct buf *b, size_t n) { size_t cap; if (n == 0) - return; + return (b->cap); if (SIZE_MAX - b->len < n) fatalx("buf_reserve: overflow"); if (b->len + n <= b->cap) - return; + return (b->cap); cap = b->cap == 0 ? BUF_CAP_INIT : b->cap; while (cap < b->len + n) { if (cap > SIZE_MAX / 2) fatalx("buf_reserve: overflow"); cap *= 2; } + return (cap); +} + +static void +buf_reserve(struct buf *b, size_t n) +{ + size_t cap; + + cap = buf_capacity_after(b, n); + if (cap == b->cap) + return; b->data = xreallocarray(b->data, cap, 1); b->cap = cap; } blob - c49211a07052b6a917c87d167d5acf4a503118ca blob + f0b1248c2891066cb0ceb7f1ecd3c7cfb05c2727 --- src/common/buf.h +++ src/common/buf.h @@ -36,6 +36,7 @@ void buf_init(struct buf *); void buf_free(struct buf *); void buf_freezero(struct buf *); void buf_reset(struct buf *); +size_t buf_capacity_after(const struct buf *, size_t); void buf_add(struct buf *, const void *, size_t); void buf_addc(struct buf *, int); void buf_addstr(struct buf *, const char *); blob - d727488aab51787284ef9447a505bfc1c8aa6e93 blob + de446bbb8a3b9c8a5a849563ab6107e5be61c24a --- src/common/http.c +++ src/common/http.c @@ -37,7 +37,15 @@ #include "http.h" #define READ_CHUNK 8192 +#define CHUNK_LINE_MAX 1024 +enum chunk_state { + CHUNK_SIZE, + CHUNK_DATA, + CHUNK_DATA_CRLF, + CHUNK_TRAILERS +}; + static void seterr(struct http *h, const char *fmt, ...) __attribute__((__format__ (printf, 2, 3))); @@ -196,11 +204,7 @@ http_connect(struct http *h, const char *host, const c struct timespec deadline; int error, s, ret, timedout; - memset(h, 0, sizeof(*h)); - h->fd = -1; - h->clen = -1; - buf_init(&h->in); - buf_init(&h->hdrs); + http_init(h); deadline_in(&deadline, timeout_ms); if (servername == NULL) @@ -293,6 +297,16 @@ fail: return (-1); } +void +http_init(struct http *h) +{ + memset(h, 0, sizeof(*h)); + h->fd = -1; + h->clen = -1; + buf_init(&h->in); + buf_init(&h->hdrs); +} + static int tls_write_all(struct http *h, const void *data, size_t len, const struct timespec *deadline) @@ -435,38 +449,72 @@ request_parts_ok(struct http *h, const unsigned char * return (1); } -int -http_request_bytes(struct http *h, const void *method, size_t methodlen, - const void *path, size_t pathlen, const void *headers, size_t headerslen, - const void *body, size_t bodylen, int timeout_ms) +static int +request_head_build(struct http *h, struct buf *req, const void *method, + size_t methodlen, const void *path, size_t pathlen, const void *headers, + size_t headerslen, int have_body, size_t bodylen) { - struct timespec deadline; - struct buf req; - int ret; - if (!request_parts_ok(h, method, methodlen, path, pathlen, headers, headerslen)) return (-1); h->request_head = methodlen == 4 && strncasecmp(method, "HEAD", methodlen) == 0; - deadline_in(&deadline, timeout_ms); /* Every caller supplies lengths; body bytes may contain NUL (I13). */ - buf_init(&req); - buf_add(&req, method, methodlen); - buf_addc(&req, ' '); - buf_add(&req, path, pathlen); - buf_addstr(&req, " HTTP/1.1\r\n"); - add_host_header(&req, h->host, h->port); - buf_addstr(&req, "Connection: close\r\n"); + buf_reset(req); + buf_add(req, method, methodlen); + buf_addc(req, ' '); + buf_add(req, path, pathlen); + buf_addstr(req, " HTTP/1.1\r\n"); + add_host_header(req, h->host, h->port); + buf_addstr(req, "Connection: close\r\n"); if (headerslen > 0) - buf_add(&req, headers, headerslen); + buf_add(req, headers, headerslen); + if (have_body) + buf_addf(req, "Content-Length: %zu\r\n", bodylen); + buf_addstr(req, "\r\n"); + return (0); +} + +int +http_request_head_build(struct http *h, struct buf *req, const void *method, + size_t methodlen, const void *path, size_t pathlen, const void *headers, + size_t headerslen, size_t bodylen) +{ + return (request_head_build(h, req, method, methodlen, path, pathlen, + headers, headerslen, 1, bodylen)); +} + +int +http_request_build(struct http *h, struct buf *req, const void *method, + size_t methodlen, const void *path, size_t pathlen, const void *headers, + size_t headerslen, const void *body, size_t bodylen) +{ + if (request_head_build(h, req, method, methodlen, path, pathlen, + headers, headerslen, body != NULL, bodylen) == -1) + return (-1); if (body != NULL) - buf_addf(&req, "Content-Length: %zu\r\n", bodylen); - buf_addstr(&req, "\r\n"); - if (body != NULL) - buf_add(&req, body, bodylen); + buf_add(req, body, bodylen); + return (0); +} +int +http_request_bytes(struct http *h, const void *method, size_t methodlen, + const void *path, size_t pathlen, const void *headers, size_t headerslen, + const void *body, size_t bodylen, int timeout_ms) +{ + struct timespec deadline; + struct buf req; + int ret; + + deadline_in(&deadline, timeout_ms); + buf_init(&req); + if (http_request_build(h, &req, method, methodlen, path, pathlen, + headers, headerslen, body, bodylen) == -1) { + buf_free(&req); + return (-1); + } + ret = tls_write_all(h, req.data, req.len, &deadline); buf_freezero(&req); /* the body may carry... nothing secret, * but the request is cheap to wipe */ @@ -630,15 +678,11 @@ response_framing(struct http *h) } int -http_response(struct http *h, int timeout_ms) +http_response_parse(struct http *h, int eof) { - struct timespec deadline; u_char *end; size_t delim, hlen, linelen; - int r; - deadline_in(&deadline, timeout_ms); - for (;;) { for (;;) { end = NULL; @@ -655,12 +699,11 @@ http_response(struct http *h, int timeout_ms) seterr(h, "response headers too large"); return (-1); } - if ((r = fill(h, &deadline)) == -1) - return (-1); - if (r == 0) { + if (eof) { seterr(h, "connection closed before headers"); return (-1); } + return (0); } hlen = (size_t)(end - h->in.data); if (hlen > HTTP_HDR_MAX) { @@ -702,9 +745,28 @@ http_response(struct http *h, int timeout_ms) h->body_eof = 1; } h->seen_headers = 1; - return (0); + return (1); } +int +http_response(struct http *h, int timeout_ms) +{ + struct timespec deadline; + int r; + + deadline_in(&deadline, timeout_ms); + for (;;) { + r = http_response_parse(h, 0); + if (r != 0) + return (r == 1 ? 0 : -1); + r = fill(h, &deadline); + if (r == -1) + return (-1); + if (r == 0) + return (http_response_parse(h, 1) == 1 ? 0 : -1); + } +} + /* * Case-insensitive header lookup in the raw block; returns the value * span (not NUL-terminated -- hostile bytes stay length-carried). @@ -788,17 +850,49 @@ parse_chunk_size(struct http *h, const u_char *line, s return (0); } +/* A trailer is an ordinary field line, but framing and hop-by-hop fields + * cannot safely acquire meaning after the response body has begun. */ +static int +parse_trailer(struct http *h, const u_char *line, size_t len) +{ + const u_char *colon; + size_t i, namelen; + + colon = memchr(line, ':', len); + if (colon == NULL || colon == line) { + seterr(h, "malformed response trailer"); + return (-1); + } + namelen = (size_t)(colon - line); + for (i = 0; i < namelen; i++) { + if (!tchar(line[i])) { + seterr(h, "malformed response trailer name"); + return (-1); + } + } + if (reserved_header(line, namelen)) { + seterr(h, "forbidden response trailer"); + return (-1); + } + for (i = namelen + 1; i < len; i++) { + if ((line[i] < 0x20 && line[i] != '\t') || line[i] == 0x7f) { + seterr(h, "malformed response trailer value"); + return (-1); + } + } + return (0); +} + /* * Stream body bytes: handles chunked and content-length framing and * read-until-close. Returns bytes produced, 0 at end of body, -1 on * error (including a peer that closes short of Content-Length). */ ssize_t -http_body(struct http *h, void *out, size_t outlen, int timeout_ms) +http_body_parse(struct http *h, void *out, size_t outlen, int eof) { - struct timespec deadline; - size_t n; - int r; + u_char *nl; + size_t linelen, n, rawlen, sz; if (!h->seen_headers) { seterr(h, "http_body before http_response"); @@ -806,18 +900,17 @@ http_body(struct http *h, void *out, size_t outlen, in } if (h->body_eof) return (0); - deadline_in(&deadline, timeout_ms); if (h->chunked) { for (;;) { - if (h->chunk_rem > 0) { + switch (h->chunk_state) { + case CHUNK_DATA: if (h->in.len == 0) { - if ((r = fill(h, &deadline)) == -1) - return (-1); - if (r == 0) { + if (eof) { seterr(h, "truncated chunk"); return (-1); } + return (HTTP_BODY_MORE); } n = h->chunk_rem; if (n > h->in.len) @@ -827,51 +920,97 @@ http_body(struct http *h, void *out, size_t outlen, in memcpy(out, h->in.data, n); buf_drain(&h->in, n); h->chunk_rem -= n; + if (h->chunk_rem == 0) + h->chunk_state = CHUNK_DATA_CRLF; return ((ssize_t)n); - } - /* need a size line (or the post-chunk CRLF) */ - for (;;) { - u_char *nl; - size_t linelen, sz; - + case CHUNK_DATA_CRLF: + if (h->in.len < 2) { + if (eof) { + seterr(h, "truncated chunk delimiter"); + return (-1); + } + return (HTTP_BODY_MORE); + } + if (h->in.data[0] != '\r' || h->in.data[1] != '\n') { + seterr(h, "missing CRLF after chunk data"); + return (-1); + } + buf_drain(&h->in, 2); + h->chunk_state = CHUNK_SIZE; + continue; + case CHUNK_SIZE: nl = h->in.len == 0 ? NULL : memchr(h->in.data, '\n', h->in.len); if (nl == NULL) { - if (h->in.len > 1024) { + if (h->in.len > CHUNK_LINE_MAX) { seterr(h, "bad chunk size"); return (-1); } - if ((r = fill(h, &deadline)) == -1) + if (eof) { + seterr(h, "truncated chunking"); return (-1); - if (r == 0) { - seterr(h, - "truncated chunking"); - return (-1); } - continue; + return (HTTP_BODY_MORE); } - linelen = (size_t)(nl - h->in.data); - if (linelen > 0 && - h->in.data[linelen - 1] == '\r') - linelen--; + rawlen = (size_t)(nl - h->in.data) + 1; + if (rawlen > CHUNK_LINE_MAX || rawlen < 2 || + nl[-1] != '\r') { + seterr(h, "bad chunk size line"); + return (-1); + } + linelen = rawlen - 2; if (linelen == 0) { - /* CRLF after a chunk's data */ - buf_drain(&h->in, - (size_t)(nl - h->in.data) + 1); - continue; + seterr(h, "bad chunk size"); + return (-1); } - if (parse_chunk_size(h, h->in.data, - linelen, &sz) == -1) + if (parse_chunk_size(h, h->in.data, linelen, &sz) == -1) return (-1); - buf_drain(&h->in, - (size_t)(nl - h->in.data) + 1); + buf_drain(&h->in, rawlen); if (sz == 0) { - /* trailers ignored to EOF */ + h->chunk_state = CHUNK_TRAILERS; + h->trailer_len = 0; + continue; + } + h->chunk_rem = sz; + h->chunk_state = CHUNK_DATA; + continue; + case CHUNK_TRAILERS: + nl = h->in.len == 0 ? NULL : + memchr(h->in.data, '\n', h->in.len); + if (nl == NULL) { + if (h->in.len > HTTP_HDR_MAX - h->trailer_len) { + seterr(h, "response trailers too large"); + return (-1); + } + if (eof) { + seterr(h, "truncated response trailers"); + return (-1); + } + return (HTTP_BODY_MORE); + } + rawlen = (size_t)(nl - h->in.data) + 1; + if (rawlen > HTTP_HDR_MAX - h->trailer_len) { + seterr(h, "response trailers too large"); + return (-1); + } + if (rawlen < 2 || nl[-1] != '\r') { + seterr(h, "malformed response trailer line"); + return (-1); + } + linelen = rawlen - 2; + if (linelen > 0 && parse_trailer(h, h->in.data, + linelen) == -1) + return (-1); + buf_drain(&h->in, rawlen); + h->trailer_len += rawlen; + if (linelen == 0) { h->body_eof = 1; return (0); } - h->chunk_rem = sz; - break; + continue; + default: + seterr(h, "invalid chunk state"); + return (-1); } } } @@ -882,9 +1021,7 @@ http_body(struct http *h, void *out, size_t outlen, in return (0); } if (h->in.len == 0) { - if ((r = fill(h, &deadline)) == -1) - return (-1); - if (r == 0) { + if (eof) { if (h->clen >= 0 && h->body_rem > 0) { seterr(h, "connection closed mid-body"); return (-1); @@ -892,6 +1029,7 @@ http_body(struct http *h, void *out, size_t outlen, in h->body_eof = 1; return (0); } + return (HTTP_BODY_MORE); } n = h->in.len; if (h->clen >= 0 && (long long)n > h->body_rem) @@ -905,6 +1043,26 @@ http_body(struct http *h, void *out, size_t outlen, in return ((ssize_t)n); } +ssize_t +http_body(struct http *h, void *out, size_t outlen, int timeout_ms) +{ + struct timespec deadline; + ssize_t n; + int r; + + deadline_in(&deadline, timeout_ms); + for (;;) { + n = http_body_parse(h, out, outlen, 0); + if (n != HTTP_BODY_MORE) + return (n); + r = fill(h, &deadline); + if (r == -1) + return (-1); + if (r == 0) + return (http_body_parse(h, out, outlen, 1)); + } +} + void http_close(struct http *h) { blob - c77d34eacd2254d4a67cddc48d55ea6861948d3f blob + c3b8eab55ab778a094365fc51da09097748e992b --- src/common/http.h +++ src/common/http.h @@ -40,6 +40,7 @@ struct tls_config; #define HTTP_HDR_MAX (64 * 1024) #define HTTP_CHUNK_MAX (16 * 1024 * 1024) +#define HTTP_BODY_MORE (-2) struct http { int fd; @@ -53,6 +54,8 @@ struct http { long long clen; /* -1: until close */ long long body_rem; size_t chunk_rem; + int chunk_state; /* incremental chunk framing state */ + size_t trailer_len; /* consumed trailer bytes, bounded */ int body_eof; int seen_headers; int request_head; /* response has no message body */ @@ -65,6 +68,11 @@ typedef int (*http_addr_ok)(const struct sockaddr *, s int http_connect(struct http *, const char *host, const char *port, const char *servername, struct tls_config *cfg, int timeout_ms, http_addr_ok addr_ok, void *arg); +void http_init(struct http *); +int http_request_build(struct http *, struct buf *, const void *, size_t, + const void *, size_t, const void *, size_t, const void *, size_t); +int http_request_head_build(struct http *, struct buf *, const void *, + size_t, const void *, size_t, const void *, size_t, size_t); int http_request(struct http *, const char *method, const char *path, const char *extra_headers, const void *body, size_t bodylen, int timeout_ms); @@ -72,10 +80,12 @@ int http_request_bytes(struct http *, const void *met const void *path, size_t, const void *headers, size_t, const void *body, size_t, int); int http_response(struct http *, int timeout_ms); +int http_response_parse(struct http *, int); int http_header(const struct http *, const char *name, const char **valp, size_t *lenp); ssize_t http_body(struct http *, void *out, size_t outlen, int timeout_ms); +ssize_t http_body_parse(struct http *, void *, size_t, int); void http_close(struct http *); const char *http_error(const struct http *); blob - /dev/null blob + 23ad5acb4b4a838aa9315c06b0bee54dbccfb07a (mode 644) --- /dev/null +++ src/common/model_window.c @@ -0,0 +1,70 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include + +#include "json.h" +#include "model_window.h" + +/* + * Model-listing formats are not uniform. Accept the common direct integer + * fields in a deterministic order, while keeping hostile JSON bounded by the + * same ceiling as context_limit. json_get_num reads from an explicit token + * span, so no provider-controlled value is treated as a C string. + */ +int64_t +model_window_parse(const struct json *j, int obj) +{ + static const char *const fields[] = { + "context_window", + "context_length", + "max_input_tokens" + }; + int64_t v; + size_t i; + int tok; + + if (!json_is_object(j, obj)) + return (0); + for (i = 0; i < sizeof(fields) / sizeof(fields[0]); i++) { + tok = json_obj_get(j, obj, fields[i]); + if (tok != -1 && json_get_num(j, tok, &v) == 0 && v > 0 && + v <= FUGU_MODEL_WINDOW_MAX) + return (v); + } + return (0); +} + +/* behavior.md section 3: operator, selected metadata, table, unknown. */ +int64_t +model_window_effective(int64_t context_limit, int64_t provider_window, + const char *model) +{ + if (context_limit > 0 && context_limit <= FUGU_MODEL_WINDOW_MAX) + return (context_limit); + if (provider_window > 0 && provider_window <= FUGU_MODEL_WINDOW_MAX) + return (provider_window); + if (model != NULL) { + if (strncmp(model, "claude-", 7) == 0) + return (200000); + if (strncmp(model, "gpt-", 4) == 0) + return (128000); + } + return (0); +} blob - /dev/null blob + c6405c749698d5784544327d362a6fcd6f030b3c (mode 644) --- /dev/null +++ src/common/model_window.h @@ -0,0 +1,30 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#ifndef MODEL_WINDOW_H +#define MODEL_WINDOW_H + +#include + +struct json; + +/* Matches the configured context_limit ceiling. */ +#define FUGU_MODEL_WINDOW_MAX INT64_C(1000000000) + +int64_t model_window_parse(const struct json *, int); +int64_t model_window_effective(int64_t, int64_t, const char *); + +#endif /* MODEL_WINDOW_H */ blob - 4f96ef1a0ba4ee0cf2a0d40ab084ff7d89af5d6e blob + af82dd62aa9b749fad477d3f73941e3ade196410 --- src/fugu/Makefile +++ src/fugu/Makefile @@ -1,8 +1,8 @@ PROG= fugu SRCS= main.c coord.c priv.c conf.c parse.y \ - journal.c sysprompt.c tooldefs.c skills.c output.c \ + journal.c sysprompt.c tooldefs.c agentcfg.c skills.c output.c \ anthropic_req.c openai_req.c msg.c \ - json.c buf.c imsgev.c log.c xmalloc.c + json.c model_window.c buf.c imsgev.c log.c xmalloc.c MAN= fugu.1 fugu.conf.5 MANDIR= ${PREFIX}/man/man blob - /dev/null blob + 68b1f752c19e603f2bbbd52fdd4da4e85438c5e3 (mode 644) --- /dev/null +++ src/fugu/agentcfg.c @@ -0,0 +1,409 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include +#include + +#include "buf.h" +#include "json.h" +#include "log.h" +#include "xmalloc.h" +#include "msg.h" +#include "conf.h" +#include "agentcfg.h" + +/* "default" is reserved for slot zero; escape a block with that name. */ +#define DEFAULT_QUALIFIER "default" +#define NAMED_DEFAULT_QUALIFIER "@default" + +struct agentcfg_provider { + char *name; /* display/route name, "default" for slot 0 */ + char *qualifier; /* unambiguous schema prefix */ + char *model; /* configured model, or NULL */ + int type; + int available; /* endpoint has a usable credential */ +}; + +struct agentcfg_choice { + char *choice; + const char *model; /* borrowed from the snapshot */ + int slot; + int type; +}; + +struct agentcfg { + struct agentcfg_provider *providers; + size_t nproviders; + struct agentcfg_choice *choices; + size_t nchoices; + char *subagent_model; + int subagent_slot; /* -1: follow the lead */ + int64_t max_subagents; + char *schema; + char *description; + struct tool_def tool; +}; + +static const struct agentcfg_provider * +provider_at(const struct agentcfg *ac, int slot) +{ + if (slot < 0 || (size_t)slot >= ac->nproviders) + return (NULL); + return (&ac->providers[slot]); +} + +/* + * Apply the common JSON codec's UTF-8 policy once at snapshot time. This + * keeps the visible enum value, exact-match key, and eventual request model + * identical even when a quoted configuration string contains invalid UTF-8. + */ +static char * +clean_string(const char *s) +{ + struct json_out jo; + struct json j; + struct buf b; + char *out; + size_t len; + + buf_init(&b); + json_out_init(&jo, &b); + json_arr_begin(&jo); + json_cstr(&jo, s); + json_arr_end(&jo); + if (json_parse(&j, b.data, b.len, 8) == -1 || + (out = json_get_str(&j, json_arr_get(&j, json_root(&j), 0), + &len)) == NULL) + fatalx("agentcfg: internal JSON round trip"); + json_done(&j); + buf_free(&b); + if (memchr(out, '\0', len) != NULL) { + free(out); + fatalx("agentcfg: NUL in configuration string"); + } + return (out); +} + +/* A direct caller cannot bypass the parser and create an enum route which + * will not fit the fixed model fields used to launch a subagent. */ +static char * +clean_model(const char *s) +{ + char *model; + size_t len; + + len = strlen(s); + if (len == 0 || len > FUGU_MODEL_ID_MAX) + return (NULL); + model = clean_string(s); + len = strlen(model); + if (len == 0 || len > FUGU_MODEL_ID_MAX) { + free(model); + return (NULL); + } + return (model); +} + +static void +clear_choices(struct agentcfg *ac) +{ + size_t i; + + for (i = 0; i < ac->nchoices; i++) + free(ac->choices[i].choice); + free(ac->choices); + ac->choices = NULL; + ac->nchoices = 0; + free(ac->schema); + ac->schema = NULL; + ac->tool.schema = NULL; +} + +/* Add a provider/model route once; equal models on different slots remain. */ +static void +add_choice(struct agentcfg *ac, int slot, int type, const char *model) +{ + const struct agentcfg_provider *p; + struct agentcfg_choice *c; + size_t i; + + if (model == NULL || (p = provider_at(ac, slot)) == NULL || + !p->available) + return; + for (i = 0; i < ac->nchoices; i++) + if (ac->choices[i].slot == slot && + strcmp(ac->choices[i].model, model) == 0) + return; + + ac->choices = xreallocarray(ac->choices, ac->nchoices + 1, + sizeof(*ac->choices)); + c = &ac->choices[ac->nchoices++]; + memset(c, 0, sizeof(*c)); + if (ac->nproviders > 1) + xasprintf(&c->choice, "%s:%s", p->qualifier, model); + else + c->choice = xstrdup(model); + c->model = model; + c->slot = slot; + c->type = type; +} + +static int +build_schema(struct agentcfg *ac, int lead_slot, int lead_type) +{ + const struct agentcfg_provider *lead; + struct json_out jo; + struct buf b; + size_t i; + int slot, type; + + if ((lead = provider_at(ac, lead_slot)) == NULL || !lead->available) + return (-1); + (void)lead; + clear_choices(ac); + + /* The immutable operator palette: implicit then named providers. */ + for (i = 0; i < ac->nproviders; i++) + add_choice(ac, (int)i, ac->providers[i].type, + ac->providers[i].model); + + /* A configured subagent model follows its provider override or lead. */ + if (ac->subagent_model != NULL) { + slot = ac->subagent_slot >= 0 ? ac->subagent_slot : lead_slot; + type = ac->subagent_slot >= 0 ? ac->providers[slot].type : + lead_type; + add_choice(ac, slot, type, ac->subagent_model); + } + + buf_init(&b); + json_out_init(&jo, &b); + json_obj_begin(&jo); + json_key(&jo, "type"); + json_cstr(&jo, "object"); + json_key(&jo, "properties"); + json_obj_begin(&jo); + json_key(&jo, "prompt"); + json_obj_begin(&jo); + json_key(&jo, "type"); + json_cstr(&jo, "string"); + json_obj_end(&jo); + json_key(&jo, "label"); + json_obj_begin(&jo); + json_key(&jo, "type"); + json_cstr(&jo, "string"); + json_key(&jo, "description"); + json_cstr(&jo, "Optional short ASCII identifier using letters, digits, " + "underscore, dash, or dot"); + json_key(&jo, "maxLength"); + json_num(&jo, FUGU_AGENT_LABEL_MAX); + json_obj_end(&jo); + json_key(&jo, "model"); + json_obj_begin(&jo); + json_key(&jo, "type"); + json_cstr(&jo, "string"); + json_key(&jo, "enum"); + json_arr_begin(&jo); + for (i = 0; i < ac->nchoices; i++) + json_cstr(&jo, ac->choices[i].choice); + json_arr_end(&jo); + json_obj_end(&jo); + json_obj_end(&jo); + json_key(&jo, "required"); + json_arr_begin(&jo); + json_cstr(&jo, "prompt"); + json_arr_end(&jo); + json_obj_end(&jo); + ac->schema = buf_cstr(&b); + ac->tool.schema = ac->schema; + return (0); +} + +struct agentcfg * +agentcfg_new(const struct fugu_conf *cf) +{ + const struct fugu_provider *fp; + struct agentcfg_provider *p; + struct agentcfg *ac; + size_t i, n = 1; + + TAILQ_FOREACH(fp, &cf->providers, entry) + n++; + ac = xcalloc(1, sizeof(*ac)); + ac->providers = xcalloc(n, sizeof(*ac->providers)); + ac->nproviders = n; + ac->subagent_slot = -1; + ac->max_subagents = cf->max_subagents; + + p = &ac->providers[0]; + p->name = xstrdup(DEFAULT_QUALIFIER); + p->qualifier = xstrdup(DEFAULT_QUALIFIER); + if (cf->model != NULL && (p->model = clean_model(cf->model)) == NULL) + goto invalid; + p->type = cf->provider_type; + /* + * parse_config marks a named slot active when slot zero was unusable. + * Check both facts because selecting that named slot also changes the + * live provider_type, which can otherwise make a legacy key look usable. + */ + p->available = cf->initial_provider_slot == 0 && + conf_effective_key(cf) != NULL; + + i = 1; + TAILQ_FOREACH(fp, &cf->providers, entry) { + p = &ac->providers[i++]; + p->name = xstrdup(fp->name); + p->qualifier = xstrdup(strcmp(fp->name, + DEFAULT_QUALIFIER) == 0 ? NAMED_DEFAULT_QUALIFIER : fp->name); + if (fp->model != NULL && (p->model = clean_model(fp->model)) == NULL) + goto invalid; + p->type = fp->type; + p->available = 1; /* parsed blocks always require an API key */ + } + + if (cf->subagent_model != NULL && + (ac->subagent_model = clean_model(cf->subagent_model)) == NULL) + goto invalid; + if (cf->subagent_provider != NULL) { + if (strcmp(cf->subagent_provider, + FUGU_IMPLICIT_PROVIDER_REF) == 0) { + if (ac->providers[0].available) + ac->subagent_slot = 0; + } else { + for (i = 1; i < ac->nproviders; i++) + if (strcmp(ac->providers[i].name, + cf->subagent_provider) == 0) { + ac->subagent_slot = (int)i; + break; + } + } + if (ac->subagent_slot == -1) { + agentcfg_free(ac); + return (NULL); + } + } + + ac->tool.name = "agent"; + xasprintf(&ac->description, + "Spawn a read-only subagent for parallel exploration, search, or " + "verification across the tree. Give it a self-contained prompt and " + "expect only its final report; it cannot modify anything. At most " + "%lld may run at once; additional calls wait for a slot. When a " + "model choice exists, use a lighter model for mechanical sweeps and " + "a stronger one for judgment-heavy verification.", + (long long)cf->max_subagents); + ac->tool.description = ac->description; + return (ac); + +invalid: + agentcfg_free(ac); + return (NULL); +} + +void +agentcfg_free(struct agentcfg *ac) +{ + size_t i; + + if (ac == NULL) + return; + clear_choices(ac); + for (i = 0; i < ac->nproviders; i++) { + free(ac->providers[i].name); + free(ac->providers[i].qualifier); + free(ac->providers[i].model); + } + free(ac->providers); + free(ac->subagent_model); + free(ac->description); + free(ac); +} + +const struct tool_def * +agentcfg_tool(struct agentcfg *ac, int lead_slot, int lead_type, + const char *lead_model) +{ + (void)lead_model; + if (ac == NULL || ac->max_subagents <= 0 || + build_schema(ac, lead_slot, lead_type) == -1) + return (NULL); + return (&ac->tool); +} + +int +agentcfg_resolve(struct agentcfg *ac, int lead_slot, int lead_type, + const char *lead_model, const char *choice, struct agentcfg_route *route) +{ + const struct agentcfg_provider *p; + char *want; + size_t i; + int slot, type; + + if (ac == NULL || route == NULL || ac->max_subagents <= 0 || + (p = provider_at(ac, lead_slot)) == NULL || !p->available) + return (-1); + if (choice != NULL) { + /* choice may itself be borrowed from the currently built enum. */ + want = xstrdup(choice); + if (build_schema(ac, lead_slot, lead_type) == -1) { + free(want); + return (-1); + } + for (i = 0; i < ac->nchoices; i++) + if (strcmp(ac->choices[i].choice, want) == 0) { + slot = ac->choices[i].slot; + p = &ac->providers[slot]; + route->provider_slot = slot; + route->provider_type = ac->choices[i].type; + route->provider_name = p->name; + route->model = ac->choices[i].model; + free(want); + return (0); + } + free(want); + return (-1); + } + + /* The two overrides fall back independently to the live lead route. */ + slot = ac->subagent_slot >= 0 ? ac->subagent_slot : lead_slot; + type = ac->subagent_slot >= 0 ? ac->providers[slot].type : lead_type; + p = &ac->providers[slot]; + if (!p->available || + (ac->subagent_model == NULL && lead_model == NULL)) + return (-1); + route->provider_slot = slot; + route->provider_type = type; + route->provider_name = p->name; + route->model = ac->subagent_model != NULL ? ac->subagent_model : + lead_model; + return (0); +} + +size_t +agentcfg_nchoices(const struct agentcfg *ac) +{ + return (ac != NULL ? ac->nchoices : 0); +} + +const char * +agentcfg_choice(const struct agentcfg *ac, size_t i) +{ + if (ac == NULL || i >= ac->nchoices) + return (NULL); + return (ac->choices[i].choice); +} blob - 9996d6c35a615be2322b7a85c13ec6ac80e23999 blob + b37a0b622352a35e2a8ded6404f13e6ccbb9447e --- src/fugu/conf.h +++ src/fugu/conf.h @@ -24,6 +24,9 @@ #define FUGU_CONF_PATH "/etc/fugu.conf" +/* Unambiguous subagent_provider spelling for the implicit provider. */ +#define FUGU_IMPLICIT_PROVIDER_REF "@default" + #include "proto.h" /* PROVIDER_* wire values */ /* blob - /dev/null blob + 38ca754b0192c4ceca0635d18080e4a3dd9d1c66 (mode 644) --- /dev/null +++ src/fugu/agentcfg.h @@ -0,0 +1,79 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#ifndef AGENTCFG_H +#define AGENTCFG_H + +#include + +struct fugu_conf; +struct tool_def; +struct agentcfg; + +/* + * A resolved subagent route. Strings are borrowed: provider_name belongs + * to agentcfg and model belongs either to agentcfg or to the lead_model + * argument. They must not be freed. provider_slot is the fugu-api slot + * (zero for the implicit provider, one-based for named provider blocks). + */ +struct agentcfg_route { + int provider_slot; + int provider_type; + const char *provider_name; + const char *model; +}; + +/* + * Snapshot the operator-configured provider/model palette. Call this + * before /model is allowed to replace cf->model. The snapshot deliberately + * copies no endpoint or credential. Credential availability is retained as + * a boolean so an unusable implicit slot cannot enter the model palette. + * NULL means the provider references or model bounds in cf are inconsistent + * (a parsed configuration is always consistent). + */ +struct agentcfg *agentcfg_new(const struct fugu_conf *); +void agentcfg_free(struct agentcfg *); + +/* + * Return the model-facing agent definition for the lead's current route. + * The returned definition and its strings are borrowed until the next call + * using this agentcfg or agentcfg_free(). NULL means max_subagents is zero + * or the lead slot is invalid. lead_model is accepted here so callers can + * use the same live-route tuple for agentcfg_resolve(); configured palette + * entries remain the immutable snapshot. With multiple provider slots the + * spelling is default:model for slot zero and name:model for a named block; + * a block literally named "default" uses @default:model. This enum spelling + * is separate from subagent_provider, where @default selects slot zero and + * default still names the literal block. These are opaque exact-match keys, + * so ':' inside model is never parsed as a separator. + */ +const struct tool_def *agentcfg_tool(struct agentcfg *, int, int, + const char *); + +/* + * Resolve an exact enum value. A NULL choice requests the configured + * default: subagent provider and model overrides fall back independently to + * the live lead route. Returns zero on success and -1 for an invalid lead + * route or a non-enum choice. No request should be made after -1. + */ +int agentcfg_resolve(struct agentcfg *, int, int, const char *, + const char *, struct agentcfg_route *); + +/* Current enum, primarily useful to request builders and focused tests. */ +size_t agentcfg_nchoices(const struct agentcfg *); +const char *agentcfg_choice(const struct agentcfg *, size_t); + +#endif /* AGENTCFG_H */ blob - 82df0b575e6d9d2d28a961f76c812ee0b4130dc9 blob + 34321c239eb0d81a4cc5b0fa3dd327f49797e757 --- src/fugu/coord.c +++ src/fugu/coord.c @@ -22,9 +22,11 @@ #include #include #include +#include #include #include #include +#include #include #include #include @@ -36,6 +38,7 @@ #include "xmalloc.h" #include "buf.h" #include "json.h" +#include "model_window.h" #include "msg.h" #include "anthropic.h" #include "openai.h" @@ -48,10 +51,12 @@ #include "sysprompt.h" #include "skills.h" #include "tooldefs.h" +#include "agentcfg.h" #define TURN_ITER_MAX 50 /* behavior.md section 5 */ #define MAX_TOOL_CALLS 64 /* tool_use blocks per assistant msg */ #define MODELS_RELAY_MAX 4096 /* picker entries per listing (I10) */ +#define AGENT_WEB_MEMORY_MAX (64 * 1024 * 1024) /* The /compact summarization instruction (behavior.md section 3). */ #define COMPACT_INSTR \ @@ -70,6 +75,8 @@ enum coord_wait { WAIT_TOOL, /* pumping one tool call to its result */ WAIT_TOOL_CTL, /* waiting for transactional job-control ACK */ WAIT_WEB, /* pumping one brokered web call to its result */ + WAIT_WEB_IDLE, /* draining a cancelled web call before another */ + WAIT_AGENTS, /* one wave of ephemeral read-only subagents */ WAIT_CTX, /* pumping a context read/append to fugu-tool */ WAIT_INPUT /* idle at the curses prompt, awaiting a UI event */ }; @@ -80,8 +87,50 @@ struct tool_call { char id[256]; char name[FUGU_TOOL_NAME_MAX]; struct buf input; + struct buf result; /* precomputed coordinator-local result (agent) */ + int result_ready; + int result_error; }; +struct coord; + +/* One on-demand fugu-tool -r instance. Its conversation and local tool + * exchanges stay inside that process; only the final report is retained. */ +struct subagent { + struct coord *coord; + struct child kid; + struct imsgev iev; + struct tool_call *tc; + int slot; /* index in coord.agents */ + int ready; + int done; + int started; + int api_inflight; + int api_assembling; + int web_building; + int web_queued; + int web_rejected; + int web_inflight; + uint32_t api_id; + uint32_t web_id; + size_t request_bytes; + size_t web_arg_bytes; + struct buf web_arg; + char web_name[FUGU_TOOL_NAME_MAX]; + struct buf out; + struct buf prompt; + char label[FUGU_AGENT_LABEL_MAX + 1]; + char model[FUGU_MODEL_ID_MAX + 1]; + int provider_type; + int provider_slot; +}; + +struct listed_model { + int slot; + int64_t context_window; + char id[FUGU_MODEL_ID_MAX + 1]; +}; + #ifndef LIBEXECDIR #define LIBEXECDIR "/usr/local/libexec" #endif @@ -129,7 +178,29 @@ struct coord { struct buf steer; /* pending steer text (2.1) */ struct buf steer_pend; /* one steer entry, being chunked */ size_t models_out; /* relayed model entries (meter) */ + struct listed_model *listed_models; /* validates picker selections */ + size_t nlisted_models; + int models_collecting; int default_provider_type; /* to restore on slot 0 */ + int active_provider_slot; + int64_t provider_window; /* selected listing metadata */ + const char *libexec; + int devnull_fd; /* pre-pledge worker stdio source */ + struct agentcfg *agentcfg; + int protect_fd; /* unlinked startup snapshot */ + uint64_t protect_len; /* NUL-separated literal paths */ + struct subagent agents[MAX_TOOL_CALLS]; + int nagents; + int agents_ready; + int agents_done; + uint32_t lead_api_id; + uint32_t cancelled_api[MAX_TOOL_CALLS]; + uint32_t cancelled_web[MAX_TOOL_CALLS]; + int ncancelled_api; + int ncancelled_web; + struct subagent *agent_web_active; + uint32_t web_busy_id; + size_t agent_web_memory; /* per-turn stream accumulation */ struct buf text; @@ -166,8 +237,118 @@ struct coord { static struct coord *coord; static void coord_teardown(struct coord *); +static void agent_dispatch(int, short, void *); +static void agent_wave_cleanup(struct coord *, int); +static void agent_web_start_next(struct coord *); +static struct subagent * +agent_by_api_id(struct coord *c, uint32_t id) +{ + int i; + + for (i = 0; i < c->nagents; i++) + if (c->agents[i].started && c->agents[i].api_id == id) + return (&c->agents[i]); + return (NULL); +} + +static struct subagent * +agent_by_web_id(struct coord *c, uint32_t id) +{ + int i; + + for (i = 0; i < c->nagents; i++) + if (c->agents[i].started && c->agents[i].web_id == id) + return (&c->agents[i]); + return (NULL); +} + +static int +cancelled_id_has(const uint32_t *ids, int nids, uint32_t id) +{ + int i; + + for (i = 0; i < nids; i++) + if (ids[i] == id) + return (1); + return (0); +} + static void +cancelled_id_add(uint32_t *ids, int *nids, uint32_t id) +{ + if (id == 0 || cancelled_id_has(ids, *nids, id)) + return; + if (*nids >= MAX_TOOL_CALLS) + fatalx("too many cancelled worker correlations"); + ids[(*nids)++] = id; +} + +static int +cancelled_id_event(uint32_t *ids, int *nids, uint32_t id, int terminal) +{ + int i; + + for (i = 0; i < *nids; i++) { + if (ids[i] != id) + continue; + if (terminal) { + memmove(&ids[i], &ids[i + 1], + (size_t)(*nids - i - 1) * sizeof(ids[0])); + (*nids)--; + } + return (1); + } + return (0); +} + +static int +agents_settled(struct coord *c) +{ + int i; + + if (c->nagents == 0) + return (1); + for (i = 0; i < c->nagents; i++) + if (!c->agents[i].done || c->agents[i].api_inflight || + c->agents[i].web_building || c->agents[i].web_queued || + c->agents[i].web_inflight) + return (0); + return (1); +} + +static uint32_t +agent_unique_id(struct coord *c) +{ + uint32_t id; + + do { + id = arc4random(); + } while (id == 0 || id == c->lead_api_id || id == c->web_id || + id == c->web_busy_id || + agent_by_api_id(c, id) != NULL || agent_by_web_id(c, id) != NULL || + cancelled_id_has(c->cancelled_api, c->ncancelled_api, id) || + cancelled_id_has(c->cancelled_web, c->ncancelled_web, id)); + return (id); +} + +/* fugu-web owns one request assembler and executes one blocking fetch at a + * time. Keep that fact explicit in the coordinator instead of relying on + * socket-buffer ordering between otherwise concurrent callers. */ +static void +web_busy_complete(struct coord *c, uint32_t id) +{ + if (id == 0 || c->web_busy_id != id) + fatalx("web busy correlation mismatch"); + c->web_busy_id = 0; + if (c->wait == WAIT_WEB_IDLE) { + c->wait = WAIT_NONE; + event_loopexit(NULL); + } + agent_web_start_next(c); +} + +static void tool_turn_ctl(struct coord *c, uint32_t type) { int rc; @@ -301,7 +482,211 @@ imsg_bytes(struct imsg *imsg, size_t *lenp) return (ibuf_data(&ib)); } +static int +protect_has(const struct buf *b, const char *path) +{ + size_t off, len; + + for (off = 0; off < b->len; off += len + 1) { + len = strnlen((const char *)b->data + off, b->len - off); + if (len == b->len - off) + fatalx("unterminated protect snapshot"); + if (strcmp((const char *)b->data + off, path) == 0) + return (1); + } + return (0); +} + static void +protect_add(struct buf *b, const char *path) +{ + if (path[0] == '\0' || protect_has(b, path)) + return; + buf_addstr(b, path); + buf_addc(b, '\0'); +} + +static int +protect_has_meta(const char *s) +{ + size_t i; + + for (i = 0; s[i] != '\0'; i++) { + if (s[i] == '\\' && s[i + 1] != '\0') { + i++; + continue; + } + if (s[i] == '*' || s[i] == '?' || s[i] == '[') + return (1); + } + return (0); +} + +/* glob(3) removes quoting backslashes from a successful match. Do the same + * for an unmatched non-pattern so a future literal `name*` remains covered + * when configured as `name\*`. */ +static char * +protect_unescape(const char *s) +{ + char *out; + size_t i, j; + + out = xmalloc(strlen(s) + 1); + for (i = j = 0; s[i] != '\0'; i++) { + if (s[i] == '\\' && s[i + 1] != '\0') + i++; + out[j++] = s[i]; + } + out[j] = '\0'; + return (out); +} + +static const char * +protect_glob_error(int error) +{ + switch (error) { + case GLOB_ABORTED: + return ("filesystem error"); + case GLOB_NOSPACE: + return ("out of memory"); + default: + return ("unknown error"); + } +} + +/* + * Materialize the startup set in one unlinked descriptor. Every persistent + * or on-demand tool role inherits it at a fixed descriptor; receivers use + * pread(2), so all forked descriptors can consume the same immutable byte + * image without sharing an offset. The file is unlinked before any + * untrusted input is consumed and therefore needs no steady-state path + * authority in the coordinator. + */ +static void +protect_snapshot_store(struct coord *c, const struct buf *snapshot) +{ + struct stat st, rost; + char path[] = "/tmp/fugu-protect.XXXXXXXXXX"; + const u_char *p = snapshot->data; + size_t left = snapshot->len; + ssize_t n; + int fd, rofd; + + if (snapshot->len == 0) + return; + if ((fd = mkstemp(path)) == -1) + fatal("mkstemp protect snapshot"); + while (left > 0) { + n = write(fd, p, left); + if (n == -1) { + if (errno == EINTR) + continue; + fatal("write protect snapshot"); + } + if (n == 0) { + errno = EIO; + fatal("write protect snapshot"); + } + p += n; + left -= (size_t)n; + } + /* Retain only a read capability after materialization. Check that the + * pathname still names the mkstemp inode before unlinking it; a same-uid + * racer can therefore cause only a fail-closed startup error. */ + if (fstat(fd, &st) == -1) + fatal("fstat protect snapshot"); + if ((rofd = open(path, O_RDONLY|O_CLOEXEC|O_NOFOLLOW)) == -1) + fatal("reopen protect snapshot read-only"); + if (fstat(rofd, &rost) == -1) + fatal("fstat read-only protect snapshot"); + if (st.st_dev != rost.st_dev || st.st_ino != rost.st_ino || + st.st_size != rost.st_size) + fatalx("protect snapshot changed while materializing"); + if (unlink(path) == -1) + fatal("unlink protect snapshot"); + close(fd); + c->protect_fd = rofd; + c->protect_len = snapshot->len; +} + +/* + * `protect` is a program-start snapshot (behavior.md 6): expand every glob + * before untrusted input and reuse these exact literal paths for persistent + * and on-demand tool roles. A literal that does not exist is retained so it + * remains hidden if created later; unmatched globs intentionally match no + * future file. + */ +static void +protect_snapshot_build(struct coord *c) +{ + struct buf snapshot; + const char *s; + + buf_init(&snapshot); + if (c->conf->protect == NULL) + goto done; + s = c->conf->protect; + while (*s != '\0') { + char *tok; + size_t n = strcspn(s, " \t\r\n"); + glob_t g; + int error; + + if (n == 0) { + s += strspn(s, " \t\r\n"); + continue; + } + tok = xmalloc(n + 1); + memcpy(tok, s, n); + tok[n] = '\0'; + memset(&g, 0, sizeof(g)); + error = glob(tok, GLOB_ERR | GLOB_NOSORT, NULL, &g); + if (error == 0) { + size_t i; + + for (i = 0; i < g.gl_pathc; i++) + protect_add(&snapshot, g.gl_pathv[i]); + } else if (error == GLOB_NOMATCH) { + if (!protect_has_meta(tok)) { + char *literal = protect_unescape(tok); + + protect_add(&snapshot, literal); + free(literal); + } + } else { + const char *reason = protect_glob_error(error); + + globfree(&g); + fatalx("protect pattern %s: %s", tok, reason); + } + globfree(&g); + free(tok); + s += n; + s += strspn(s, " \t\r\n"); + } + protect_snapshot_store(c, &snapshot); +done: + buf_free(&snapshot); +} + +static void +send_tool_hello(struct coord *c, struct imsgev *iev, int agent) +{ + struct hello_tool h; + + memset(&h, 0, sizeof(h)); + h.allow_write = agent ? 0 : c->conf->allow_write; + h.allow_subprocess_net = agent ? 0 : c->conf->allow_subprocess_net; + h.max_bg_jobs = agent ? 0 : (int)c->conf->max_bg_jobs; + h.agent = agent; + h.protect_snapshot = 1; + h.protect_len = c->protect_len; + if (imsgev_compose(iev, FUGU_IMSG_HELLO, 0, -1, -1, &h, + sizeof(h)) == -1) + fatal("compose tool HELLO"); +} + +static void send_hello(struct coord *c, enum fugu_role role) { struct imsgev *iev = &c->iev[role]; @@ -434,38 +819,7 @@ send_hello(struct coord *c, enum fugu_role role) fatal("compose editor HELLO"); break; case ROLE_TOOL: { - struct buf b; - struct hello_tool h; - - memset(&h, 0, sizeof(h)); - h.allow_write = c->conf->allow_write; - h.allow_subprocess_net = c->conf->allow_subprocess_net; - h.max_bg_jobs = (int)c->conf->max_bg_jobs; - - buf_init(&b); - buf_add(&b, &h, sizeof(h)); - /* protect: whitespace-separated in config, NUL-separated - * on the wire so a glob may hold spaces after expansion */ - if (c->conf->protect != NULL) { - const char *s = c->conf->protect; - - while (*s != '\0') { - size_t n = strcspn(s, " \t\r\n"); - - if (n > 0) { - buf_add(&b, s, n); - buf_addc(&b, '\0'); - } - s += n; - s += strspn(s, " \t\r\n"); - } - } - if (imsgev_compose(iev, FUGU_IMSG_HELLO, 0, -1, -1, - b.data, b.len) == -1) { - buf_free(&b); - fatal("compose tool HELLO"); - } - buf_free(&b); + send_tool_hello(c, iev, 0); break; } default: @@ -727,6 +1081,19 @@ usage_add(struct a_usage *dst, const struct a_usage *s return (0); } +static int +usage_account(struct coord *c, const struct a_usage *u) +{ + struct a_usage session = c->usage; + struct a_usage turn = c->turn_usage; + + if (usage_add(&session, u) == -1 || usage_add(&turn, u) == -1) + return (-1); + c->usage = session; + c->turn_usage = turn; + return (0); +} + static void stream_turn_fail(struct coord *c, const char *reason) { @@ -780,6 +1147,8 @@ stream_event(struct coord *c, struct imsg *imsg) */ if (c->wait != WAIT_STREAM) return; + if (imsg_get_id(imsg) != c->lead_api_id) + fatalx("lead provider correlation mismatch"); p = imsg_bytes(imsg, &len); @@ -813,8 +1182,7 @@ stream_event(struct coord *c, struct imsg *imsg) memcpy(&u, p, sizeof(u)); if (c->turn_data_exceeded) break; - if (usage_add(&c->usage, &u) == -1 || - usage_add(&c->turn_usage, &u) == -1) { + if (usage_account(c, &u) == -1) { /* This is a whole-Turn aggregate failure. Drain the * current uncorrelated response through its terminal. */ c->turn_data_exceeded = 1; @@ -853,6 +1221,9 @@ stream_event(struct coord *c, struct imsg *imsg) memcpy(tc->name, name, namelen); tc->name[namelen] = '\0'; buf_init(&tc->input); + buf_init(&tc->result); + tc->result_ready = 0; + tc->result_error = 0; break; } case FUGU_IMSG_A_TOOL_INPUT: { @@ -1010,37 +1381,36 @@ web_event(struct coord *c, struct imsg *imsg) if (p == NULL || len == 0 || len > FUGU_WEB_OUT_MAX - c->tool_out.len) { c->turn_data_exceeded = 1; - c->tool_done = 1; - c->wait = WAIT_NONE; - event_loopexit(NULL); + c->tool_is_error = 1; break; } - buf_add(&c->tool_out, p, len); + if (!c->turn_data_exceeded) + buf_add(&c->tool_out, p, len); break; case FUGU_IMSG_WEB_RESULT: { struct tool_result r; if (p == NULL || len != sizeof(r)) { c->turn_data_exceeded = 1; - c->tool_done = 1; - c->wait = WAIT_NONE; - event_loopexit(NULL); - break; + c->tool_is_error = 1; + } else { + memcpy(&r, p, sizeof(r)); + if (r.is_error != 0 && r.is_error != 1) { + c->turn_data_exceeded = 1; + c->tool_is_error = 1; + } else + c->tool_is_error = r.is_error; } - memcpy(&r, p, sizeof(r)); - if (r.is_error != 0 && r.is_error != 1) { - c->turn_data_exceeded = 1; - c->tool_done = 1; - c->wait = WAIT_NONE; - event_loopexit(NULL); - break; - } - c->tool_is_error = r.is_error; + if (c->turn_data_exceeded) + c->tool_is_error = 1; c->tool_done = 1; c->wait = WAIT_NONE; + web_busy_complete(c, imsg_get_id(imsg)); event_loopexit(NULL); break; } + default: + fatalx("bad web event"); } } @@ -1090,6 +1460,34 @@ switch_id_ok(const u_char *p, size_t len) return (1); } +static const struct listed_model * +listed_model_find(const struct coord *c, int slot, const char *id) +{ + size_t i; + + for (i = 0; i < c->nlisted_models; i++) + if (c->listed_models[i].slot == slot && + strcmp(c->listed_models[i].id, id) == 0) + return (&c->listed_models[i]); + return (NULL); +} + +static const char * +configured_provider_name(const struct coord *c, int slot) +{ + const struct fugu_provider *fp; + int i = 0; + + if (slot == 0) + return (""); + if (slot < 0) + return (NULL); + TAILQ_FOREACH(fp, &c->conf->providers, entry) + if (++i == slot) + return (fp->name); + return (NULL); +} + /* Input events from the curses front end. */ static void ui_event(struct coord *c, struct imsg *imsg) @@ -1121,7 +1519,9 @@ ui_event(struct coord *c, struct imsg *imsg) case FUGU_IMSG_UI_QUIT: c->want_quit = 1; if (c->wait == WAIT_STREAM || c->wait == WAIT_TOOL || - c->wait == WAIT_TOOL_CTL || c->wait == WAIT_WEB) { + c->wait == WAIT_TOOL_CTL || c->wait == WAIT_WEB || + c->wait == WAIT_WEB_IDLE || + c->wait == WAIT_AGENTS) { c->cancelled = 1; c->wait = WAIT_NONE; event_loopexit(NULL); @@ -1167,7 +1567,8 @@ ui_event(struct coord *c, struct imsg *imsg) c->want_switch = 1; if (c->wait == WAIT_INPUT || c->wait == WAIT_STREAM || c->wait == WAIT_TOOL || c->wait == WAIT_TOOL_CTL || - c->wait == WAIT_WEB) { + c->wait == WAIT_WEB || c->wait == WAIT_WEB_IDLE || + c->wait == WAIT_AGENTS) { c->wait = WAIT_NONE; event_loopexit(NULL); } @@ -1175,6 +1576,7 @@ ui_event(struct coord *c, struct imsg *imsg) case FUGU_IMSG_UI_SELECT_MODEL: { struct model_sel sel; struct fugu_provider *fp; + const struct listed_model *listed; char id[FUGU_MODEL_ID_MAX + 1]; int i; @@ -1184,10 +1586,16 @@ ui_event(struct coord *c, struct imsg *imsg) len - sizeof(sel) > FUGU_MODEL_ID_MAX) fatalx("bad model selection"); memcpy(&sel, p, sizeof(sel)); + if (sel.context_window < 0 || + sel.context_window > FUGU_MODEL_WINDOW_MAX) + fatalx("bad model context window"); memcpy(id, p + sizeof(sel), len - sizeof(sel)); id[len - sizeof(sel)] = '\0'; if (strlen(id) != len - sizeof(sel)) fatalx("NUL in model id"); /* I13 */ + if ((listed = listed_model_find(c, sel.slot, id)) == NULL) + fatalx("model selection was not in the provider listing"); + sel.context_window = listed->context_window; /* slot 0 restores the default provider; slot N is the Nth * named block, whose provider becomes active with it */ @@ -1205,6 +1613,8 @@ ui_event(struct coord *c, struct imsg *imsg) fatalx("bad provider slot"); imsgev_compose(&c->iev[ROLE_API], FUGU_IMSG_SET_PROVIDER, 0, -1, -1, &sel, sizeof(sel)); + c->active_provider_slot = sel.slot; + c->provider_window = sel.context_window; free(c->conf->model); c->conf->model = xstrdup(id); c->prev_msgs = 0; /* the cache starts cold (8) */ @@ -1227,19 +1637,63 @@ model_event(struct coord *c, struct imsg *imsg) p = imsg_bytes(imsg, &len); switch (imsg_get_type(imsg)) { case FUGU_IMSG_MODEL: - if (p == NULL || len <= sizeof(struct model_ent) || + { + struct model_ent me; + struct listed_model *listed; + const u_char *name, *nul, *id; + const char *expected; + size_t i, rest, namelen, idlen; + + if (!c->models_collecting || p == NULL || + len <= sizeof(struct model_ent) || len > sizeof(struct model_ent) + 64 + FUGU_MODEL_ID_MAX) fatalx("bad MODEL entry"); + memcpy(&me, p, sizeof(me)); + if (me.slot < 0 || me.context_window < 0 || + me.context_window > FUGU_MODEL_WINDOW_MAX) + fatalx("bad MODEL metadata"); + name = p + sizeof(me); + rest = len - sizeof(me); + if ((nul = memchr(name, '\0', rest)) == NULL) + fatalx("bad MODEL provider name"); + namelen = (size_t)(nul - name); + expected = configured_provider_name(c, me.slot); + if (expected == NULL || strlen(expected) != namelen || + memcmp(name, expected, namelen) != 0) + fatalx("MODEL provider does not match its slot"); + id = nul + 1; + idlen = (size_t)(name + rest - id); + if (idlen == 0 || idlen > FUGU_MODEL_ID_MAX || + memchr(id, '\0', idlen) != NULL) + fatalx("bad MODEL id"); + for (i = 0; i < c->nlisted_models; i++) + if (c->listed_models[i].slot == me.slot && + strlen(c->listed_models[i].id) == idlen && + memcmp(c->listed_models[i].id, id, idlen) == 0) + break; + if (i != c->nlisted_models) + break; /* one authoritative window per provider/model */ /* A hostile provider controls its listing length; past the * relay cap, stop forwarding (the picker stays usable) rather * than crash the session -- this is flood defense, not a * protocol violation. */ if (c->models_out >= MODELS_RELAY_MAX) break; + c->listed_models = xreallocarray(c->listed_models, + c->nlisted_models + 1, sizeof(*c->listed_models)); + listed = &c->listed_models[c->nlisted_models++]; + memset(listed, 0, sizeof(*listed)); + listed->slot = me.slot; + listed->context_window = me.context_window; + memcpy(listed->id, id, idlen); c->models_out++; ui_send(c, FUGU_IMSG_UI_MODEL, p, len); break; + } case FUGU_IMSG_MODELS_END: + if (!c->models_collecting) + fatalx("unexpected MODELS_END"); + c->models_collecting = 0; c->models_out = 0; ui_send(c, FUGU_IMSG_UI_MODELS_END, NULL, 0); break; @@ -1344,6 +1798,128 @@ allowed_from(enum fugu_role role, uint32_t type) } static void +agent_web_arg_free(struct coord *c, struct subagent *a) +{ + if (a->web_arg.cap > c->agent_web_memory) + fatalx("subagent web memory accounting underflow"); + c->agent_web_memory -= a->web_arg.cap; + buf_free(&a->web_arg); +} + +static void +agent_web_local_error(struct subagent *a, const char *s) +{ + struct tool_result r; + + if (a->done) + return; + if (imsgev_compose(&a->iev, FUGU_IMSG_AGENT_WEB_OUT, 0, -1, -1, + s, strlen(s)) == -1) + fatal("compose subagent web local error"); + memset(&r, 0, sizeof(r)); + r.is_error = 1; + if (imsgev_compose(&a->iev, FUGU_IMSG_AGENT_WEB_RESULT, 0, -1, -1, + &r, sizeof(r)) == -1) + fatal("compose subagent web local result"); +} + +static void +agent_web_start_next(struct coord *c) +{ + struct subagent *a; + struct tool_req req; + size_t off; + int i; + + if (c->agent_web_active != NULL || c->web_busy_id != 0) + return; + for (i = 0; i < c->nagents; i++) { + a = &c->agents[i]; + if (!a->web_queued) + continue; + a->web_queued = 0; + if (a->done) { + agent_web_arg_free(c, a); + continue; + } + memset(&req, 0, sizeof(req)); + strlcpy(req.name, a->web_name, sizeof(req.name)); + a->web_id = agent_unique_id(c); + a->web_inflight = 1; + c->agent_web_active = a; + c->web_busy_id = a->web_id; + if (imsgev_compose(&c->iev[ROLE_WEB], FUGU_IMSG_WEB_REQ, + a->web_id, -1, -1, &req, sizeof(req)) == -1) + fatal("relay subagent WEB_REQ"); + for (off = 0; off < a->web_arg.len; ) { + size_t k = a->web_arg.len - off; + + if (k > FUGU_CHUNK) + k = FUGU_CHUNK; + if (imsgev_compose(&c->iev[ROLE_WEB], + FUGU_IMSG_WEB_ARG, a->web_id, -1, -1, + a->web_arg.data + off, k) == -1) + fatal("relay subagent WEB_ARG"); + off += k; + } + if (imsgev_compose(&c->iev[ROLE_WEB], FUGU_IMSG_WEB_RUN, + a->web_id, -1, -1, NULL, 0) == -1) + fatal("relay subagent WEB_RUN"); + agent_web_arg_free(c, a); + return; + } +} + +static void +agent_provider_event(struct subagent *a, struct imsg *imsg) +{ + const u_char *p; + size_t len; + uint32_t type = imsg_get_type(imsg); + + if (!a->api_inflight) + fatalx("provider event for idle subagent"); + p = imsg_bytes(imsg, &len); + if (!a->done && imsgev_compose(&a->iev, type, 0, -1, -1, p, len) == -1) + fatal("relay provider event to subagent"); + if (type == FUGU_IMSG_A_DONE || type == FUGU_IMSG_A_ERROR) + a->api_inflight = 0; + if (a->coord->wait == WAIT_AGENTS && agents_settled(a->coord)) + event_loopexit(NULL); +} + +static void +agent_web_event(struct subagent *a, struct imsg *imsg) +{ + struct coord *c = a->coord; + const u_char *p; + size_t len; + uint32_t outtype; + + if (!a->web_inflight || c->agent_web_active != a || + c->web_busy_id != a->web_id || imsg_get_id(imsg) != a->web_id) + fatalx("web event for idle subagent"); + p = imsg_bytes(imsg, &len); + if (imsg_get_type(imsg) == FUGU_IMSG_WEB_OUT) + outtype = FUGU_IMSG_AGENT_WEB_OUT; + else if (imsg_get_type(imsg) == FUGU_IMSG_WEB_RESULT) + outtype = FUGU_IMSG_AGENT_WEB_RESULT; + else + fatalx("bad subagent web event"); + if (!a->done && imsgev_compose(&a->iev, outtype, 0, -1, -1, p, + len) == -1) + fatal("relay web event to subagent"); + if (outtype == FUGU_IMSG_AGENT_WEB_RESULT) { + a->web_inflight = 0; + a->web_id = 0; + c->agent_web_active = NULL; + web_busy_complete(c, imsg_get_id(imsg)); + } + if (c->wait == WAIT_AGENTS && agents_settled(c)) + event_loopexit(NULL); +} + +static void coord_dispatch(int fd, short event, void *arg) { struct imsgev *iev = arg; @@ -1426,7 +2002,21 @@ coord_dispatch(int fd, short event, void *arg) case FUGU_IMSG_A_RETRY: case FUGU_IMSG_A_DONE: case FUGU_IMSG_A_ERROR: - stream_event(coord, &imsg); + if (imsg_get_id(&imsg) == coord->lead_api_id) + stream_event(coord, &imsg); + else { + struct subagent *a; + int terminal; + + terminal = type == FUGU_IMSG_A_DONE || + type == FUGU_IMSG_A_ERROR; + if ((a = agent_by_api_id(coord, + imsg_get_id(&imsg))) != NULL) + agent_provider_event(a, &imsg); + else if (!cancelled_id_event(coord->cancelled_api, + &coord->ncancelled_api, imsg_get_id(&imsg), terminal)) + fatalx("unknown provider correlation id"); + } break; case FUGU_IMSG_TOOL_OUT: case FUGU_IMSG_TOOL_DIFF: @@ -1438,7 +2028,27 @@ coord_dispatch(int fd, short event, void *arg) break; case FUGU_IMSG_WEB_OUT: case FUGU_IMSG_WEB_RESULT: - web_event(coord, &imsg); + if (imsg_get_id(&imsg) == coord->web_id) + web_event(coord, &imsg); + else { + struct subagent *a; + int terminal = type == FUGU_IMSG_WEB_RESULT; + int cancelled; + uint32_t id = imsg_get_id(&imsg); + + if ((a = agent_by_web_id(coord, + id)) != NULL) + agent_web_event(a, &imsg); + else { + cancelled = cancelled_id_event( + coord->cancelled_web, + &coord->ncancelled_web, id, terminal); + if (!cancelled) + fatalx("unknown web correlation id"); + if (terminal) + web_busy_complete(coord, id); + } + } break; case FUGU_IMSG_CTX_DATA: case FUGU_IMSG_CTX_END: @@ -1473,9 +2083,10 @@ coord_dispatch(int fd, short event, void *arg) } static void -send_request(struct coord *c, struct buf *body) +send_request(struct coord *c, struct buf *body, uint32_t id, int provider_slot) { struct imsgev *iev = &c->iev[ROLE_API]; + struct api_request_end end; size_t off; for (off = 0; off < body->len; ) { @@ -1483,11 +2094,14 @@ send_request(struct coord *c, struct buf *body) if (n > FUGU_CHUNK) n = FUGU_CHUNK; - imsgev_compose(iev, FUGU_IMSG_REQUEST, 0, -1, -1, + imsgev_compose(iev, FUGU_IMSG_REQUEST, id, -1, -1, body->data + off, n); off += n; } - imsgev_compose(iev, FUGU_IMSG_REQUEST_END, 0, -1, -1, NULL, 0); + memset(&end, 0, sizeof(end)); + end.provider_slot = provider_slot; + imsgev_compose(iev, FUGU_IMSG_REQUEST_END, id, -1, -1, &end, + sizeof(end)); } /* Send one tool call to fugu-tool and pump events until its result. */ @@ -1529,12 +2143,31 @@ run_web(struct coord *c, struct tool_call *tc) struct imsgev *iev = &c->iev[ROLE_WEB]; struct tool_req req; size_t off; + int rc; + buf_reset(&c->tool_out); + buf_reset(&c->diff); + c->tool_done = 0; + c->tool_is_error = 0; + while (c->web_busy_id != 0 && !c->cancelled && !c->want_switch && + !c->want_quit) { + c->wait = WAIT_WEB_IDLE; + rc = event_dispatch(); + if (rc == -1) + fatal("event_dispatch web drain"); + if (rc == 1 && c->web_busy_id != 0) + fatalx("web drain lost its event source"); + } + c->wait = WAIT_NONE; + if (c->cancelled || c->want_switch || c->want_quit) { + c->tool_is_error = 1; + return; + } + memset(&req, 0, sizeof(req)); strlcpy(req.name, tc->name, sizeof(req.name)); - do { - c->web_id = arc4random(); - } while (c->web_id == 0); + c->web_id = agent_unique_id(c); + c->web_busy_id = c->web_id; if (imsgev_compose(iev, FUGU_IMSG_WEB_REQ, c->web_id, -1, -1, &req, sizeof(req)) == -1) fatal("compose WEB_REQ"); @@ -1552,24 +2185,269 @@ run_web(struct coord *c, struct tool_call *tc) NULL, 0) == -1) fatal("compose WEB_RUN"); - buf_reset(&c->tool_out); - buf_reset(&c->diff); - c->tool_done = 0; - c->tool_is_error = 0; c->wait = WAIT_WEB; - event_dispatch(); + rc = event_dispatch(); + if (rc == -1) + fatal("event_dispatch web"); c->wait = WAIT_NONE; + if (!c->tool_done) { + cancelled_id_add(c->cancelled_web, &c->ncancelled_web, + c->web_id); + if (rc == 1) + fatalx("web call lost its event source"); + } c->web_id = 0; } +static void +agent_failed(struct subagent *a, const char *reason) +{ + struct coord *c = a->coord; + + if (a->done) + return; + if (a->web_building || a->web_queued || a->web_rejected) { + a->web_building = 0; + a->web_queued = 0; + a->web_rejected = 0; + a->web_arg_bytes = 0; + agent_web_arg_free(c, a); + } + if (a->api_assembling || a->api_inflight) { + if (imsgev_compose(&c->iev[ROLE_API], + FUGU_IMSG_REQUEST_CANCEL, a->api_id, -1, -1, + NULL, 0) == -1) + fatal("compose failed-agent REQUEST_CANCEL"); + a->api_assembling = 0; + a->api_inflight = 1; /* cleared by the cancellation terminal */ + } + buf_reset(&a->out); + buf_addf(&a->out, "subagent failed: %s", reason); + buf_reset(&a->tc->result); + buf_add(&a->tc->result, a->out.data, a->out.len); + a->tc->result_ready = 1; + a->tc->result_error = 1; + a->done = 1; + c->agents_done++; + agent_web_start_next(c); + if (c->wait == WAIT_AGENTS && agents_settled(c)) + event_loopexit(NULL); +} + +/* Dynamic fugu-tool -r channel. It may request only provider/web brokering + * and return progress/usage/final-report frames; every other authority is a + * protocol violation (I10/I15). */ +static void +agent_dispatch(int fd, short event, void *arg) +{ + struct subagent *a = arg; + struct coord *c = a->coord; + struct imsgbuf *ibuf = &a->iev.ibuf; + struct imsg imsg; + const u_char *p; + struct tool_req req; + ssize_t n; + size_t len, off; + uint32_t type; + + (void)fd; + if (event & EV_READ) { + if ((n = imsgbuf_read(ibuf)) == -1) + fatal("subagent imsgbuf_read"); + if (n == 0) { + if (!a->ready) { + a->ready = 1; + c->agents_ready++; + } + if (!a->done) + agent_failed(a, "worker exited before its report"); + if (c->wait == WAIT_AGENTS && c->agents_ready == c->nagents) + event_loopexit(NULL); + imsgev_clear(&a->iev); + a->kid.fd = -1; + return; + } + } + if (event & EV_WRITE) { + if (imsgbuf_write(ibuf) == -1) { + if (errno == EPIPE) { + if (!a->ready) { + a->ready = 1; + c->agents_ready++; + } + agent_failed(a, "worker channel closed"); + if (c->wait == WAIT_AGENTS && + c->agents_ready == c->nagents) + event_loopexit(NULL); + imsgev_clear(&a->iev); + a->kid.fd = -1; + return; + } + fatal("subagent imsgbuf_write"); + } + } + + for (;;) { + if ((n = imsg_get(ibuf, &imsg)) == -1) + fatal("subagent imsg_get"); + if (n == 0) + break; + type = imsg_get_type(&imsg); + p = imsg_bytes(&imsg, &len); + switch (type) { + case FUGU_IMSG_READY: + if (len != 0 || a->ready) + fatalx("bad subagent READY"); + a->ready = 1; + c->agents_ready++; + if (c->wait == WAIT_AGENTS && c->agents_ready == c->nagents) + event_loopexit(NULL); + break; + case FUGU_IMSG_AGENT_REQUEST: + if (!a->started || a->done || a->api_inflight || + len == 0 || len > FUGU_REQUEST_MAX - a->request_bytes) + fatalx("bad subagent provider request chunk"); + a->api_assembling = 1; + a->request_bytes += len; + if (imsgev_compose(&c->iev[ROLE_API], FUGU_IMSG_REQUEST, + a->api_id, -1, -1, p, len) == -1) + fatal("relay subagent provider request"); + break; + case FUGU_IMSG_AGENT_REQUEST_END: { + struct api_request_end end; + + if (!a->api_assembling || a->api_inflight || len != 0) + fatalx("bad subagent provider request end"); + memset(&end, 0, sizeof(end)); + end.provider_slot = a->provider_slot; + if (imsgev_compose(&c->iev[ROLE_API], + FUGU_IMSG_REQUEST_END, a->api_id, -1, -1, &end, + sizeof(end)) == -1) + fatal("relay subagent request end"); + a->api_assembling = 0; + a->api_inflight = 1; + a->request_bytes = 0; + break; + } + case FUGU_IMSG_AGENT_WEB_REQ: + if (!a->started || a->done || a->web_building || + a->web_queued || a->web_rejected || + a->web_inflight || p == NULL || len != sizeof(req)) + fatalx("bad subagent web request"); + memcpy(&req, p, sizeof(req)); + if (memchr(req.name, '\0', sizeof(req.name)) == NULL) + fatalx("unterminated subagent web tool"); + strlcpy(a->web_name, req.name, sizeof(a->web_name)); + agent_web_arg_free(c, a); + a->web_arg_bytes = 0; + a->web_building = 1; + break; + case FUGU_IMSG_AGENT_WEB_ARG: { + size_t cap, growth; + + if (a->done || !a->web_building || p == NULL || len == 0 || + len > FUGU_WEB_ARG_MAX - a->web_arg_bytes) + fatalx("bad subagent web arguments"); + a->web_arg_bytes += len; + if (a->web_rejected) + break; + cap = buf_capacity_after(&a->web_arg, len); + growth = cap - a->web_arg.cap; + if (growth > AGENT_WEB_MEMORY_MAX - c->agent_web_memory) { + a->web_rejected = 1; + agent_web_arg_free(c, a); + break; + } + buf_add(&a->web_arg, p, len); + c->agent_web_memory += growth; + break; + } + case FUGU_IMSG_AGENT_WEB_RUN: + if (a->done || !a->web_building || len != 0) + fatalx("bad subagent web run"); + a->web_building = 0; + a->web_arg_bytes = 0; + if (a->web_rejected) { + a->web_rejected = 0; + agent_web_arg_free(c, a); + agent_web_local_error(a, + "web request rejected: broker queue memory limit"); + break; + } + a->web_queued = 1; + agent_web_start_next(c); + break; + case FUGU_IMSG_AGENT_USAGE: { + struct a_usage u; + + if (p == NULL || len != sizeof(u)) + fatalx("bad subagent usage"); + memcpy(&u, p, sizeof(u)); + if (usage_account(c, &u) == -1) + c->turn_data_exceeded = 1; + else + machine_usage(c, &u); + break; + } + case FUGU_IMSG_AGENT_PROGRESS: + if (p == NULL || len == 0 || len > 256) + fatalx("bad subagent progress"); + for (off = 0; off < len; off++) + if (p[off] < 0x20 || p[off] > 0x7e) + fatalx("unsafe subagent progress"); + emit_msg(c, FUGU_IMSG_UI_NOTE, stderr, "agent[%s]: %.*s", + a->label, (int)len, p); + break; + case FUGU_IMSG_AGENT_OUT: + if (p == NULL || len == 0 || len > FUGU_TOOL_OUT_MAX - a->out.len) + fatalx("subagent report overrun"); + buf_add(&a->out, p, len); + break; + case FUGU_IMSG_AGENT_RESULT: { + struct agent_result r; + + if (p == NULL || len != sizeof(r) || a->done || + a->api_assembling || a->api_inflight || + a->request_bytes != 0 || a->web_building || + a->web_queued || a->web_rejected || a->web_inflight || + a->web_arg_bytes != 0) + fatalx("bad subagent result"); + memcpy(&r, p, sizeof(r)); + if ((r.is_error != 0 && r.is_error != 1) || + (r.truncated != 0 && r.truncated != 1) || + r.iterations < 0 || r.iterations > 25) + fatalx("invalid subagent result"); + buf_reset(&a->tc->result); + buf_add(&a->tc->result, a->out.data, a->out.len); + a->tc->result_ready = 1; + a->tc->result_error = r.is_error; + a->done = 1; + c->agents_done++; + emit_msg(c, FUGU_IMSG_UI_NOTE, stderr, + "agent[%s]: completed%s", a->label, + r.truncated ? " (truncated)" : ""); + if (c->wait == WAIT_AGENTS && agents_settled(c)) + event_loopexit(NULL); + break; + } + default: + fatalx("invalid subagent imsg %u", type); + } + imsg_free(&imsg); + } + imsgev_add(&a->iev); +} + /* Free the assembled tool_use blocks between stream pumps. */ static void reset_tcalls(struct coord *c) { int i; - for (i = 0; i < c->ntcalls; i++) + for (i = 0; i < c->ntcalls; i++) { buf_free(&c->tcalls[i].input); + buf_free(&c->tcalls[i].result); + } c->ntcalls = 0; } @@ -1749,11 +2627,23 @@ build_provider_request(struct coord *c, struct buf *bo static int send_turn_request(struct coord *c) { - const struct tool_def *tools; + const struct tool_def *base, *agent; + struct tool_def tools[20]; struct buf body; - int ntools; + int nbase, ntools; - tools = tooldefs(c->conf, c->skills, &ntools); + base = tooldefs(c->conf, c->skills, &nbase); + if (nbase > (int)(sizeof(tools) / sizeof(tools[0]))) + fatalx("too many tool definitions"); + memcpy(tools, base, (size_t)nbase * sizeof(tools[0])); + ntools = nbase; + agent = agentcfg_tool(c->agentcfg, c->active_provider_slot, + c->conf->provider_type, c->conf->model); + if (agent != NULL) { + if (ntools >= (int)(sizeof(tools) / sizeof(tools[0]))) + fatalx("too many tool definitions"); + tools[ntools++] = *agent; + } buf_init(&body); build_provider_request(c, &body, tools, ntools, c->prev_msgs > 0 ? c->prev_msgs - 1 : -1); @@ -1772,7 +2662,7 @@ send_turn_request(struct coord *c) FUGU_REQUEST_MAX / (1024 * 1024)); return (-1); } - send_request(c, &body); + send_request(c, &body, c->lead_api_id, c->active_provider_slot); buf_free(&body); c->prev_msgs = msg_count(&c->conv); /* messages just sent */ return (0); @@ -1867,6 +2757,380 @@ account_tool_result(struct coord *c) return (0); } +static void agent_call_error(struct tool_call *, const char *, ...) + __attribute__((__format__ (printf, 2, 3))); + +static void +agent_call_error(struct tool_call *tc, const char *fmt, ...) +{ + va_list ap; + + buf_reset(&tc->result); + va_start(ap, fmt); + buf_addvf(&tc->result, fmt, ap); + va_end(ap); + tc->result_ready = 1; + tc->result_error = 1; +} + +static int +agent_label_ok(const char *s, size_t len) +{ + size_t i; + + for (i = 0; i < len; i++) + if (!((s[i] >= 'a' && s[i] <= 'z') || + (s[i] >= 'A' && s[i] <= 'Z') || + (s[i] >= '0' && s[i] <= '9') || s[i] == '_' || + s[i] == '-' || s[i] == '.')) + return (0); + return (1); +} + +static int +agent_prepare(struct coord *c, struct subagent *a, struct tool_call *tc, + int ordinal) +{ + struct agentcfg_route route; + struct json j; + char *prompt = NULL, *label = NULL, *choice = NULL; + size_t promptlen = 0, labellen = 0, choicelen = 0; + int root, t; + + memset(a, 0, sizeof(*a)); + a->coord = c; + a->tc = tc; + a->kid.fd = -1; + buf_init(&a->web_arg); + buf_init(&a->out); + buf_init(&a->prompt); + if (c->conf->max_subagents <= 0) { + agent_call_error(tc, "agent: subagents are disabled"); + goto bad; + } + if (json_parse(&j, tc->input.data, tc->input.len, 0) != 0) { + agent_call_error(tc, "agent: malformed input"); + goto bad; + } + root = json_root(&j); + if (!json_is_object(&j, root) || + (t = json_obj_get(&j, root, "prompt")) == -1 || + (prompt = json_get_str(&j, t, &promptlen)) == NULL) { + agent_call_error(tc, "agent: a prompt string is required"); + goto jsonbad; + } + if ((t = json_obj_get(&j, root, "label")) != -1 && + (label = json_get_str(&j, t, &labellen)) == NULL) { + agent_call_error(tc, "agent: label must be a string"); + goto jsonbad; + } + if ((t = json_obj_get(&j, root, "model")) != -1 && + (choice = json_get_str(&j, t, &choicelen)) == NULL) { + agent_call_error(tc, "agent: model must be a palette string"); + goto jsonbad; + } + if (label != NULL && (strlen(label) != labellen || + labellen > FUGU_AGENT_LABEL_MAX || + !agent_label_ok(label, labellen))) { + agent_call_error(tc, "agent: label must be a short ASCII identifier"); + goto jsonbad; + } + if (choice != NULL && strlen(choice) != choicelen) { + agent_call_error(tc, "agent: model contains a NUL byte"); + goto jsonbad; + } + if (promptlen > FUGU_TURN_DATA_MAX) { + agent_call_error(tc, "agent: prompt exceeds 4 MiB"); + goto jsonbad; + } + if (agentcfg_resolve(c->agentcfg, c->active_provider_slot, + c->conf->provider_type, c->conf->model, choice, &route) == -1) { + agent_call_error(tc, "agent: model is outside the configured palette"); + goto jsonbad; + } + if (route.model == NULL || strlen(route.model) == 0 || + strlen(route.model) > FUGU_MODEL_ID_MAX) { + agent_call_error(tc, "agent: routed model id is invalid"); + goto jsonbad; + } + a->provider_slot = route.provider_slot; + a->provider_type = route.provider_type; + strlcpy(a->model, route.model, sizeof(a->model)); + if (label != NULL && labellen > 0) + strlcpy(a->label, label, sizeof(a->label)); + else + (void)snprintf(a->label, sizeof(a->label), "agent-%d", ordinal); + if (promptlen > 0) + buf_add(&a->prompt, prompt, promptlen); + free(prompt); + free(label); + free(choice); + json_done(&j); + + if (priv_spawn(&a->kid, c->libexec, ROLE_TOOL, "fugu-tool", 0, + !c->ui || c->debug, c->verbose, 1, c->devnull_fd, + c->protect_len > 0 ? c->protect_fd : -1) == -1) { + agent_call_error(tc, "agent: cannot spawn read-only worker"); + goto bad; + } + a->api_id = agent_unique_id(c); + a->slot = c->nagents; + a->started = 1; + imsgev_init(&a->iev, a->kid.fd, agent_dispatch, a); + send_tool_hello(c, &a->iev, 1); + return (0); + +jsonbad: + free(prompt); + free(label); + free(choice); + json_done(&j); +bad: + buf_free(&a->web_arg); + buf_free(&a->out); + buf_free(&a->prompt); + return (-1); +} + +static void +agent_system(struct coord *c, struct buf *out) +{ + const char *base = c->conf->system != NULL ? c->conf->system : + fugu_system_prompt; + static const char guidance[] = + "\n\nYou are an ephemeral read-only subagent. Work only on the " + "self-contained task you were given. Explore and verify with the " + "available read-only tools; you cannot modify files or start another " + "agent. Return a concise final report with evidence, uncertainty, and " + "useful file:line references. Do not address the user directly."; + + buf_reset(out); + buf_addstr(out, base); + if (c->ctx_personal.len > 0) { + buf_addstr(out, "\n\n"); + buf_add(out, c->ctx_personal.data, c->ctx_personal.len); + } + if (c->ctx_project.len > 0) { + buf_addstr(out, "\n\n"); + buf_add(out, c->ctx_project.data, c->ctx_project.len); + } + buf_add(out, guidance, sizeof(guidance) - 1); +} + +static void +agent_compose_bytes(struct imsgev *iev, uint32_t type, const void *data, + size_t len) +{ + const u_char *p = data; + size_t off, n; + + for (off = 0; off < len; off += n) { + n = len - off; + if (n > FUGU_CHUNK) + n = FUGU_CHUNK; + if (imsgev_compose(iev, type, 0, -1, -1, p + off, n) == -1) + fatal("compose subagent launch bytes"); + } +} + +static void +agent_start(struct coord *c, struct subagent *a, const struct buf *system) +{ + struct agent_start start; + struct buf b; + + memset(&start, 0, sizeof(start)); + start.provider_type = a->provider_type; + start.provider_slot = a->provider_slot; + start.web_search = c->conf->web_search; + start.http_request = c->conf->http_allow != NULL; + start.max_tokens = c->conf->max_tokens; + buf_init(&b); + buf_add(&b, &start, sizeof(start)); + buf_addstr(&b, a->model); + buf_addc(&b, '\0'); + buf_addstr(&b, a->label); + buf_addc(&b, '\0'); + if (imsgev_compose(&a->iev, FUGU_IMSG_AGENT_START, 0, -1, -1, + b.data, b.len) == -1) + fatal("compose AGENT_START"); + buf_free(&b); + agent_compose_bytes(&a->iev, FUGU_IMSG_AGENT_SYSTEM, system->data, + system->len); + agent_compose_bytes(&a->iev, FUGU_IMSG_AGENT_PROMPT, a->prompt.data, + a->prompt.len); + if (imsgev_compose(&a->iev, FUGU_IMSG_AGENT_RUN, 0, -1, -1, + NULL, 0) == -1) + fatal("compose AGENT_RUN"); +} + +static void +agent_wave_cleanup(struct coord *c, int cancel) +{ + struct timespec pause = { 0, 10000000 }; + int alive[MAX_TOOL_CALLS]; + pid_t pid; + int i, j, remaining = 0, status; + + memset(alive, 0, sizeof(alive)); + /* First revoke every channel and signal every child. No wedged worker + * gets a private one-second grace period: the whole wave shares one. */ + for (i = 0; i < c->nagents; i++) { + struct subagent *a = &c->agents[i]; + + if (cancel && (a->api_assembling || a->api_inflight)) { + cancelled_id_add(c->cancelled_api, &c->ncancelled_api, + a->api_id); + if (imsgev_compose(&c->iev[ROLE_API], + FUGU_IMSG_REQUEST_CANCEL, a->api_id, -1, -1, + NULL, 0) == -1) + fatal("compose subagent REQUEST_CANCEL"); + a->api_assembling = 0; + a->api_inflight = 0; + } + if (cancel && a->web_inflight) { + cancelled_id_add(c->cancelled_web, &c->ncancelled_web, + a->web_id); + if (c->agent_web_active != a || + c->web_busy_id != a->web_id) + fatalx("cancelled subagent web state mismatch"); + c->agent_web_active = NULL; + a->web_inflight = 0; + a->web_id = 0; + } + if (a->web_building || a->web_queued || a->web_rejected) { + a->web_building = 0; + a->web_queued = 0; + a->web_rejected = 0; + a->web_arg_bytes = 0; + agent_web_arg_free(c, a); + } + if (a->kid.fd >= 0) { + if (cancel) { + (void)imsgev_compose(&a->iev, FUGU_IMSG_SHUTDOWN, 0, + -1, -1, NULL, 0); + (void)imsgbuf_flush(&a->iev.ibuf); + (void)kill(a->kid.pid, SIGTERM); + } + imsgev_clear(&a->iev); + a->kid.fd = -1; + } + if (a->kid.pid > 0) { + alive[i] = 1; + remaining++; + } + } + + for (j = 0; j < 100 && remaining > 0; j++) { + for (i = 0; i < c->nagents; i++) { + struct subagent *a = &c->agents[i]; + + if (!alive[i]) + continue; + pid = waitpid(a->kid.pid, &status, WNOHANG); + if (pid == a->kid.pid || (pid == -1 && errno == ECHILD)) { + alive[i] = 0; + remaining--; + } + } + if (remaining > 0) + (void)nanosleep(&pause, NULL); + } + for (i = 0; i < c->nagents; i++) + if (alive[i]) + (void)kill(c->agents[i].kid.pid, SIGKILL); + for (i = 0; i < c->nagents; i++) { + struct subagent *a = &c->agents[i]; + + if (alive[i]) + while (waitpid(a->kid.pid, &status, 0) == -1 && errno == EINTR) + ; + agent_web_arg_free(c, a); + buf_free(&a->out); + buf_free(&a->prompt); + a->started = 0; + } + c->nagents = 0; + c->agents_ready = 0; + c->agents_done = 0; + if (c->agent_web_active != NULL || c->agent_web_memory != 0) + fatalx("subagent web state survived wave cleanup"); +} + +/* Run all agent calls in waves bounded by max_subagents. Each wave is live + * concurrently; excess calls wait for the next slot as advertised. */ +static void +run_pending_agents(struct coord *c) +{ + struct buf system; + int cap, found, i, rc; + + buf_init(&system); + agent_system(c, &system); + cap = (int)c->conf->max_subagents; + if (cap > MAX_TOOL_CALLS) + cap = MAX_TOOL_CALLS; + for (;;) { + found = 0; + c->nagents = 0; + c->agents_ready = 0; + c->agents_done = 0; + for (i = 0; i < c->ntcalls; i++) { + struct tool_call *tc = &c->tcalls[i]; + + if (strcmp(tc->name, "agent") != 0 || tc->result_ready) + continue; + found = 1; + if (cap == 0) { + agent_call_error(tc, "agent: subagents are disabled"); + continue; + } + if (c->nagents >= cap) + continue; /* queued for the next wave */ + if (agent_prepare(c, &c->agents[c->nagents], tc, i + 1) == 0) + c->nagents++; + } + if (!found) + break; + if (c->nagents == 0) + continue; /* this pass resolved only input errors */ + + while (c->agents_ready < c->nagents && !c->cancelled && + !c->want_switch && !c->want_quit) { + c->wait = WAIT_AGENTS; + rc = event_dispatch(); + if (rc == -1) + fatal("event_dispatch subagent startup"); + if (rc == 1 && c->agents_ready < c->nagents) + fatalx("subagent startup lost its event source"); + } + c->wait = WAIT_NONE; + if (c->cancelled || c->want_switch || c->want_quit) { + agent_wave_cleanup(c, 1); + break; + } + for (i = 0; i < c->nagents; i++) + if (!c->agents[i].done) + agent_start(c, &c->agents[i], &system); + + while (!agents_settled(c) && !c->cancelled && !c->want_switch && + !c->want_quit) { + c->wait = WAIT_AGENTS; + rc = event_dispatch(); + if (rc == -1) + fatal("event_dispatch subagent wave"); + if (rc == 1 && !agents_settled(c)) + fatalx("subagent wave lost its event source"); + } + c->wait = WAIT_NONE; + agent_wave_cleanup(c, + c->cancelled || c->want_switch || c->want_quit); + if (c->cancelled || c->want_switch || c->want_quit) + break; + } + buf_free(&system); +} + /* Run every pending tool_use, returning a user message of results. */ static struct msg * run_pending_tools(struct coord *c) @@ -1874,14 +3138,29 @@ run_pending_tools(struct coord *c) struct msg *um; int i; + run_pending_agents(c); um = msg_new(ROLE_USER); + if (c->cancelled || c->want_switch || c->want_quit) + return (um); for (i = 0; i < c->ntcalls; i++) { struct tool_call *tc = &c->tcalls[i]; c->turn_tool_calls++; machine_tool_call(c, tc); emit_tool(c, tc->name); /* terse (S1) */ - if (strcmp(tc->name, "skill") == 0) { + if (strcmp(tc->name, "agent") == 0) { + if (!tc->result_ready) + agent_call_error(tc, "agent: no result"); + buf_reset(&c->tool_out); + if (tc->result.len > 0) + buf_add(&c->tool_out, tc->result.data, tc->result.len); + if (account_tool_result(c) == -1) + break; + msg_add_tool_result(um, tc->id, + (const char *)tc->result.data, tc->result.len, + tc->result_error); + machine_tool_result(c, tc, &tc->result, tc->result_error); + } else if (strcmp(tc->name, "skill") == 0) { int err; resolve_skill(c, tc, &c->tool_out, &err); @@ -2253,7 +3532,7 @@ cmd_compact(struct coord *c) emit_error(c, "fugu: too large to compact"); return (1); } - send_request(c, &body); + send_request(c, &body, c->lead_api_id, c->active_provider_slot); buf_free(&body); c->turn_data = 0; c->turn_data_exceeded = 0; @@ -2351,17 +3630,8 @@ est_tokens(size_t bytes) static int64_t effective_window(struct coord *c) { - const char *m = c->conf->model; - - if (c->conf->context_limit > 0) - return (c->conf->context_limit); - if (m != NULL) { /* a small compiled table */ - if (strncmp(m, "claude-", 7) == 0) - return (200000); - if (strncmp(m, "gpt-", 4) == 0) - return (128000); - } - return (0); /* unknown: omit percentages */ + return (model_window_effective(c->conf->context_limit, + c->provider_window, c->conf->model)); } /* @@ -2481,6 +3751,10 @@ line_command(struct coord *c, const char *line, int *t * mode politely refuses (2.2). */ if (c->ui) { + free(c->listed_models); + c->listed_models = NULL; + c->nlisted_models = 0; + c->models_collecting = 1; imsgev_compose(&c->iev[ROLE_API], FUGU_IMSG_LIST_MODELS, 0, -1, -1, NULL, 0); emit_note(c, "fetching the model list..."); @@ -2728,6 +4002,8 @@ coord_teardown(struct coord *c) pid_t pid; int i, live, role, status; + if (c->nagents > 0) + agent_wave_cleanup(c, 1); for (role = ROLE_TTY; role < ROLE_MAX; role++) { if (c->kids[role].pid <= 0) continue; @@ -2814,7 +4090,13 @@ coord_run(struct fugu_conf *cf, const char *prompt, si c.debug = debug; c.verbose = verbose; c.output_mode = output_mode; + c.devnull_fd = -1; + c.protect_fd = -1; c.default_provider_type = cf->provider_type; /* restore on slot 0 */ + c.active_provider_slot = cf->initial_provider_slot; + c.agentcfg = agentcfg_new(cf); + if (c.agentcfg == NULL) + fatalx("cannot build subagent model palette"); coord = &c; /* terminal mode (curses) iff there is no one-shot prompt and both * stdin and stdout are a tty; decided before HELLO so fugu-tty knows @@ -2835,11 +4117,18 @@ coord_run(struct fugu_conf *cf, const char *prompt, si buf_init(&c.steer_pend); TAILQ_INIT(&c.conv); libexec = libexec_dir(); + c.libexec = libexec; + do { + c.lead_api_id = arc4random(); + } while (c.lead_api_id == 0); if (signal(SIGPIPE, SIG_IGN) == SIG_ERR) fatal("signal"); event_init(); + protect_snapshot_build(&c); + if ((c.devnull_fd = priv_open_devnull()) == -1) + fatal("open /dev/null"); /* Spawn every worker first (needs exec), before the journal fd * exists, so no worker ever inherits it (asset A3). */ @@ -2848,7 +4137,9 @@ coord_run(struct fugu_conf *cf, const char *prompt, si if (priv_spawn(kid, libexec, WORKERS[i].role, WORKERS[i].title, WORKERS[i].pass_tty, - 1, c.verbose, 0) == -1) + 1, c.verbose, 0, c.devnull_fd, + WORKERS[i].role == ROLE_TOOL && c.protect_len > 0 ? + c.protect_fd : -1) == -1) fatalx("cannot spawn %s", WORKERS[i].title); imsgev_init(&c.iev[WORKERS[i].role], kid->fd, coord_dispatch, &c.iev[WORKERS[i].role]); @@ -2929,6 +4220,12 @@ coord_run(struct fugu_conf *cf, const char *prompt, si signal_del(&c.sigint_ev); coord_teardown(&c); journal_close(c.journal); + close(c.devnull_fd); + c.devnull_fd = -1; + if (c.protect_fd != -1) { + close(c.protect_fd); + c.protect_fd = -1; + } /* * A drawer switch lands here with every worker reaped (fugu-tty @@ -2958,9 +4255,11 @@ coord_run(struct fugu_conf *cf, const char *prompt, si } skills_free(c.skills); + agentcfg_free(c.agentcfg); msglist_free(&c.conv); reset_tcalls(&c); free(c.sessions); + free(c.listed_models); buf_free(&c.text); buf_free(&c.output); buf_free(&c.tool_out); blob - ca7e0d61764d27865b45fc325a951c05a13bf6b4 blob + 5973654b6acc47a01ff59f5b6554a413238a662b --- src/fugu/fugu.1 +++ src/fugu/fugu.1 @@ -395,6 +395,116 @@ Each command group is anchored by a supervisor tied to an abandoned turn is not reported complete until its pending jobs have been removed. . +.Pp +When +.Cm max_subagents +is non-zero, the lead model is offered an +.Cm agent +tool with the input +.Li "{prompt, label?, model?}" . +The prompt is required and starts a fresh, ephemeral conversation. +The optional label identifies progress messages; otherwise +.Nm +assigns a label such as +.Dq agent-1 . +Prompts are limited to 4 MiB and final reports to 1 MiB. +Labels are at most 63 bytes and contain only ASCII letters, digits, +underscores, hyphens, and dots. +The subagent receives the configured system prompt and current personal and +project context, but none of the lead conversation or compaction summary. +Only its final report is returned to the lead. +. +.Pp +Subagents are confined read-only independently of +.Cm allow_write +and +.Cm allow_subprocess_net . +They may use +.Cm read , grep , find , ls , +and a read-only, networkless +.Cm shell . +They may also use +.Cm web_search +and +.Cm web_fetch +when web search is enabled, and +.Cm http_request +when an HTTP allowlist is configured. +They are never offered mutating or background-job tools, +.Cm skill , +or +.Cm agent ; +subagents therefore cannot nest. +Model and web credentials remain in their normal custodian processes and are +not sent to the subagent. +. +.Pp +The optional +.Cm model +argument is an exact member of the operator-configured palette shown in the +tool schema. +With only the implicit provider its entries are bare model identifiers. +When named providers exist, an implicit-provider entry is +.Li default:model , +a named-provider entry is +.Li name:model , +and a provider block literally named +.Dq default +uses +.Li @default:model . +These strings are opaque exact-match keys; a colon inside a model identifier +is not parsed as a separator. +The palette contains the configured model of each provider and the configured +.Cm subagent_model , +with duplicate routes removed. +An implicit provider without an effective API key is omitted. +Omitting +.Cm model +uses +.Cm subagent_provider +and +.Cm subagent_model +where configured; each missing setting falls back independently to the lead's +active provider or model. +A value outside the palette produces an error result without making a provider +request. +. +.Pp +Several +.Cm agent +calls in one response run in waves of at most +.Cm max_subagents ; +excess calls wait for the next wave and results retain tool-call order. +The read-only worker processes in a wave run concurrently. +Their model requests retain independent correlated DNS, TLS, HTTP, retry, and +stream state inside the single API credential custodian, so a slow connection +does not stall another subagent. +Brokered web calls pass through the separate web custodian; the coordinator +queues them and admits one complete request at a time. +Each subagent is limited to 25 tool rounds and the configured +.Cm max_tokens +for each generation. +At the tool-round bound it receives one final tool-free request to conclude; +the returned report is marked truncated. +Provider length stops retain text already emitted, close any dangling tool +calls without executing them, and take the same single tool-free conclusion +attempt before returning a truncated report. +If retained prose exceeds the report bound, truncation preserves the final +conclusion or terminal error diagnostic as well as a useful prefix. +. +.Pp +Line and print modes write live +.Li agent[label] +progress to standard error, while curses displays the same notices in the +transcript. +A print-mode interrupt, quitting, or switching conversations during the parent +turn terminates every running subagent and does not start a later wave. +Subagent conversations and their local tool exchanges are not journaled. +The lead's +.Cm agent +call and returned report are ordinary transactional tool-exchange records and +are replayed on resume. +. .Sh PROVIDERS Anthropic Messages and OpenAI-compatible chat-completions providers use streaming HTTPS and may invoke tools. @@ -526,8 +636,11 @@ source. Within the starting project directory, the model can read, modify, or delete anything not covered by .Cm protect . -The protect set is expanded once at startup; later-created matching files are -not covered, and protected names may still appear in directory listings. +The protect set is expanded once at startup. +An unmatched glob covers no path created later, and protected names may still +appear in directory listings. +An explicitly named literal path is retained even when absent and remains +blocked if it is created later. Starting .Nm in @@ -580,6 +693,22 @@ The web worker cannot write files or execute programs, are confined to the API worker. . .Pp +Each subagent runs in a fresh tool role whose own steady pledge omits write, +create, and network promises and whose project unveil is locked +read-and-execute only. +The subprocess exec promises admit the write and file-attribute classes that +.Xr ksh 1 +needs to establish its own startup pledge, but still omit network and unveil +authority. +The inherited read-only unveil remains the effective filesystem ceiling, so a +subprocess cannot write or create a project file despite those pledge classes. +Subagent web access remains brokered, and the role receives neither provider +nor web credentials. +The same startup-expanded +.Cm protect +paths are unveiled away from every persistent and ephemeral tool role. +. +.Pp .Pa /etc/fugu.conf and every included or key file must not be world-accessible or group-writable and must be owned by root or by the invoking user. blob - 01eec80c151c494f92002a2c24e896b1e6f7f708 blob + 4f69669b6fb8484f887f3b214d145c03139eeecd --- src/fugu/fugu.conf.5 +++ src/fugu/fugu.conf.5 @@ -138,6 +138,7 @@ The default is .Ic no . This widens only subprocess confinement; fugu's file and web tool roles do not inherit it. +Read-only subagent subprocesses do not inherit it either. .It Cm allow_write Ic yes | no Offer file mutation tools and give the tool executor kernel write and create permissions in the project directory. @@ -212,11 +213,17 @@ Set the maximum number of concurrent background shell The value must be between 0 and 256 and defaults to 4. Zero withholds the background-job tools. .It Cm max_subagents Ar number -Reserve the concurrent subagent limit for the M9 subagent integration. +Set the maximum number of read-only subagent workers in one wave. The value must be between 0 and 256 and defaults to 4. -The current build parses and reports this setting but does not yet offer the +Zero withholds the .Cm agent -tool, regardless of its value. +tool. +When one model response requests more subagents than this limit, excess calls +wait for a later wave and their results retain tool-call order. +Workers in a wave and their correlated model connections run concurrently +while credentials remain confined to the single API custodian. +Their separately brokered web calls are queued by the coordinator and admitted +to the web custodian one complete request at a time. .It Cm max_tokens Ar number Set the maximum output tokens requested for one model turn. The value must be between 1 and 100000000 and defaults to 4096. @@ -241,6 +248,8 @@ Set whitespace-separated project-relative paths or patterns whose contents tools, subprocesses, and subagents cannot reach. The patterns are expanded once at startup and unveiled with no permissions before model input is processed. +An unmatched glob covers no path created later; a literal path is retained +even when it does not yet exist. Protecting a directory covers its subtree. Names may remain visible in directory listings. .It Cm provider Ar type @@ -255,11 +264,33 @@ Select the web-search backend. The supported value and default are .Cm kagi . .It Cm subagent_model Ar model -Reserve the default subagent model for M9. -The current build parses and reports this value but does not act on it. +Set the model used by an +.Cm agent +call that omits its +.Cm model +argument. +By default it is unset and the lead's active model is used. +This model is also added to the model palette advertised by the +.Cm agent +tool. .It Cm subagent_provider Ar name -Reserve the named provider for the M9 subagent model. -The current build parses and reports this value but does not act on it. +Set the provider used by an +.Cm agent +call that omits its +.Cm model +argument. +The value must identify a configured named provider block, or be +.Dq @default +to select the implicit provider. +The latter is rejected unless the implicit provider has an effective API key. +A named block literally named +.Dq default +is selected by +.Dq default , +without the at sign. +By default it is unset and the lead's active provider is used. +Provider and model fallback are independent: setting only one directive keeps +the lead's active value for the other. .It Cm system Ar prompt Replace the built-in system prompt with one quoted line. This value composes with personal and project context; it does not disable @@ -330,6 +361,23 @@ block is active at startup. Selecting a model also selects its provider for the remainder of the session. . .Pp +Configured provider models form the +.Cm agent +tool's per-call model palette. +With no named blocks, the implicit model appears as a bare identifier. +When named blocks exist, the implicit-provider spelling is +.Li default:model +and a named-provider spelling is +.Li name:model . +A block literally named +.Dq default +uses +.Li @default:model . +These are exact enum values, not strings split at the colon; model identifiers +may therefore contain colons. +An implicit provider without an effective API key is omitted from the palette. +. +.Pp The following directives are accepted inside a named provider block: .Bl -tag -width Ds .It Cm type Cm anthropic | openai @@ -394,6 +442,11 @@ provider "work" { api_key_file "/etc/fugu-work.key" } +# Offer the read-only agent tool and use the work endpoint by default. +max_subagents 4 +subagent_provider "work" +subagent_model "gpt-5" + match group "contractors" { allow_write no web_search no @@ -417,8 +470,8 @@ Use narrow host and path patterns and explicit blocks. .Pp .Cm protect is a startup snapshot rather than a continuing policy engine. -Files created after startup that would have matched a pattern are not -automatically protected. +Files created after startup that would have matched an unmatched glob are not +automatically protected; an explicitly named literal path remains blocked. Keep credentials outside all project trees instead of relying on this list. . .Pp blob - b353e62e83b9c1c5c510c66bed77a23fc4f5fe05 blob + 5f810745077e13d960888de5c7afbfdce0812cad --- src/fugu/parse.y +++ src/fugu/parse.y @@ -114,6 +114,7 @@ static void set_bool(int *, int); static void free_secret(char *); static char *read_keyfile(const char *); static int parse_provider_type(const char *); +static int valid_model_id(const char *, const char *); static int valid_provider_name(const char *); static int provider_wire_ok(const char *, const char *, const char *, const char *); @@ -185,8 +186,7 @@ varset : STRING '=' STRING { ; main : MODEL STRING { - if (strlen($2) == 0) { - yyerror("empty model"); + if (!valid_model_id("model", $2)) { free($2); YYERROR; } @@ -246,6 +246,10 @@ main : MODEL STRING { set_number(&conf->max_subagents, $2); } | SUBAGENTMODEL STRING { + if (!valid_model_id("subagent_model", $2)) { + free($2); + YYERROR; + } set_string(&conf->subagent_model, $2); } | SUBAGENTPROVIDER STRING { @@ -462,8 +466,7 @@ provopt : TYPE STRING { prov->type = type; } | MODEL STRING { - if (strlen($2) == 0) { - yyerror("empty model"); + if (!valid_model_id("model", $2)) { free($2); YYERROR; } @@ -1009,11 +1012,23 @@ parse_config(const char *filename, struct fugu_conf *x if (errors == 0 && conf->subagent_provider != NULL) { struct fugu_provider *p; - TAILQ_FOREACH(p, &conf->providers, entry) { - if (strcmp(p->name, conf->subagent_provider) == 0) - break; + p = NULL; + if (strcmp(conf->subagent_provider, + FUGU_IMPLICIT_PROVIDER_REF) == 0) { + if (conf_effective_key(conf) == NULL) { + log_warnx("subagent_provider %s: implicit provider " + "has no API key", conf->subagent_provider); + errors++; + } + } else { + TAILQ_FOREACH(p, &conf->providers, entry) { + if (strcmp(p->name, + conf->subagent_provider) == 0) + break; + } } - if (p == NULL) { + if (strcmp(conf->subagent_provider, + FUGU_IMPLICIT_PROVIDER_REF) != 0 && p == NULL) { log_warnx("subagent_provider %s: no such provider", conf->subagent_provider); errors++; @@ -1238,6 +1253,24 @@ parse_provider_type(const char *s) return (-1); } +/* Every configured model may cross fixed API/UI/subagent frames. */ +static int +valid_model_id(const char *what, const char *s) +{ + size_t len; + + len = strlen(s); + if (len == 0) { + yyerror("empty %s", what); + return (0); + } + if (len > FUGU_MODEL_ID_MAX) { + yyerror("%s exceeds %d bytes", what, FUGU_MODEL_ID_MAX); + return (0); + } + return (1); +} + /* * Provider block names join the agent tool's model palette as * "name:model" enum strings; keep them to a charset that cannot be blob - 6c280c9aecd9e0d9e8c5964f2dd30ab103080750 blob + 2b0f2f6a9fcc6155df7323c5b8a4e22eb67e1a5d --- src/fugu/priv.c +++ src/fugu/priv.c @@ -33,10 +33,28 @@ extern char **environ; +/* Keep this above the fixed control fd so the child can install fd 3 first. */ int +priv_open_devnull(void) +{ + int fd, nfd; + + if ((fd = open(_PATH_DEVNULL, O_RDWR|O_CLOEXEC)) == -1) + return (-1); + if (fd > FUGU_CTL_FD) + return (fd); + if ((nfd = fcntl(fd, F_DUPFD_CLOEXEC, FUGU_CTL_FD + 1)) == -1) { + close(fd); + return (-1); + } + close(fd); + return (nfd); +} + +int priv_spawn(struct child *c, const char *libexec, enum fugu_role role, const char *title, int pass_tty, int keep_stderr, int log_verbose, - int readonly) + int readonly, int devnull_fd, int protect_fd) { char path[1024]; char *argv[6]; @@ -69,13 +87,6 @@ priv_spawn(struct child *c, const char *libexec, enum if (signal(SIGINT, SIG_IGN) == SIG_ERR) fatal("signal SIGINT"); - /* pin our coordinator channel to the well-known fd */ - if (fds[1] != FUGU_CTL_FD) { - if (dup2(fds[1], FUGU_CTL_FD) == -1) - fatal("dup2 ctl fd"); - } else if (fcntl(fds[1], F_SETFD, 0) == -1) - fatal("fcntl"); /* clear CLOEXEC in place */ - /* * Redirect stdin/stdout to /dev/null unless this role owns * the terminal. Keep inherited stderr when diagnostics are @@ -84,7 +95,9 @@ priv_spawn(struct child *c, const char *libexec, enum * other descriptor (style.md section 2.4). */ if (!pass_tty) { - if ((devnull = open(_PATH_DEVNULL, O_RDWR)) == -1) + devnull = devnull_fd; + if (devnull == -1 && + (devnull = open(_PATH_DEVNULL, O_RDWR)) == -1) fatal("open %s", _PATH_DEVNULL); if (dup2(devnull, STDIN_FILENO) == -1 || dup2(devnull, STDOUT_FILENO) == -1 || @@ -94,8 +107,28 @@ priv_spawn(struct child *c, const char *libexec, enum if (devnull > STDERR_FILENO) close(devnull); } - closefrom(FUGU_CTL_FD + 1); + /* A tool role may inherit the unlinked startup protect snapshot. + * Install it before fd 3, since the coordinator's original snapshot + * descriptor may itself be fd 3. */ + if (protect_fd != -1) { + if (protect_fd != FUGU_PROTECT_FD && + dup2(protect_fd, FUGU_PROTECT_FD) == -1) + fatal("dup2 protect fd"); + if (fcntl(FUGU_PROTECT_FD, F_SETFD, 0) == -1) + fatal("fcntl protect fd"); + } + + /* pin our coordinator channel to the well-known fd */ + if (fds[1] != FUGU_CTL_FD) { + if (dup2(fds[1], FUGU_CTL_FD) == -1) + fatal("dup2 ctl fd"); + } else if (fcntl(fds[1], F_SETFD, 0) == -1) + fatal("fcntl"); /* clear CLOEXEC in place */ + + closefrom(protect_fd == -1 ? FUGU_CTL_FD + 1 : + FUGU_PROTECT_FD + 1); + argc = 0; argv[argc++] = (char *)title; if (keep_stderr) blob - 082d90cb7c2411dcf6a945cf581d4db611ef8682 blob + 7273ba385a212b7b8456fc0b3d81199d3570d442 --- src/fugu/priv.h +++ src/fugu/priv.h @@ -45,10 +45,15 @@ struct child { * terminal (fugu-tty and fugu-editor). Otherwise stdin/stdout go to * /dev/null; keep_stderr leaves stderr inherited for line/print mode * and for curses debugging. log_verbose is forwarded as -v/-vv. - * readonly appends "-r" (fugu-tool subagent role). + * readonly appends "-r" (fugu-tool subagent role). devnull_fd may name a + * pre-opened O_RDWR /dev/null descriptor; -1 opens it in the child, which is + * only valid before the caller has dropped path promises. protect_fd, when + * non-negative, is inherited readably at FUGU_PROTECT_FD and every other + * descriptor above the control channel is closed. */ +int priv_open_devnull(void); int priv_spawn(struct child *c, const char *libexec, enum fugu_role, const char *title, int pass_tty, int keep_stderr, int log_verbose, - int readonly); + int readonly, int devnull_fd, int protect_fd); #endif /* PRIV_H */ blob - 94fdca85b55f01f9a8f863dedbd458e8948cb53f blob + 750ea902fc8e0d16176f26e7f41b4c95e6aefdd8 --- src/fugu/proto.h +++ src/fugu/proto.h @@ -35,6 +35,10 @@ * fugu-web -> coordinator READY, results (M8) * coordinator -> fugu-tool HELLO(cwd,protect), tool calls (M5), SHUTDOWN * fugu-tool -> coordinator READY, tool results (M5) + * coordinator -> fugu-tool -r HELLO, agent task and brokered provider/web + * events (M9), SHUTDOWN + * fugu-tool -r -> coordinator READY, provider/web requests, progress and + * the final ephemeral report (M9) * coordinator -> fugu-editor HELLO(empty), compose seed (M7), SHUTDOWN * fugu-editor -> coordinator READY, compose result (M7) * coordinator -> every worker LOG_SYSLOG once curses owns the tty (M10) @@ -47,6 +51,9 @@ /* The fixed descriptor each worker's coordinator channel is pinned * to before execve (the vmd PROC_PARENT_SOCK_FILENO idiom). */ #define FUGU_CTL_FD 3 +/* A tool worker with a non-empty startup protect set inherits its unlinked + * snapshot here. Every other worker closes this descriptor before exec. */ +#define FUGU_PROTECT_FD (FUGU_CTL_FD + 1) /* Provider protocol types, as they cross the wire in hello_api and * hello_prov (behavior.md section 8). */ @@ -112,6 +119,7 @@ * sides: provider listings are hostile input (I13). */ #define FUGU_MODEL_ID_MAX 128 #define FUGU_PROVIDER_NAME_MAX 63 +#define FUGU_AGENT_LABEL_MAX 63 #define FUGU_API_HOST_MAX 255 #define FUGU_API_PORT_MAX 15 #define FUGU_API_PATH_MAX 255 @@ -147,6 +155,7 @@ enum fugu_imsg { * an overrun as a protocol violation. */ FUGU_IMSG_REQUEST, /* one body chunk (the projection) */ FUGU_IMSG_REQUEST_END, /* body complete; run the request */ + FUGU_IMSG_REQUEST_CANCEL, /* cancel one correlated request */ FUGU_IMSG_A_TEXT, /* assistant text delta */ FUGU_IMSG_A_TOOL_BEGIN, /* tool_use opened: struct a_tool_begin + id\0name */ FUGU_IMSG_A_TOOL_INPUT, /* partial tool input JSON (with index prefix) */ @@ -264,6 +273,29 @@ enum fugu_imsg { FUGU_IMSG_WEB_OUT, /* web -> coordinator: one result chunk */ FUGU_IMSG_WEB_RESULT, /* web -> coordinator: struct tool_result */ + /* + * One ephemeral subagent (behavior.md 5.12, invariants.md I15). + * The process is the existing fugu-tool binary in its -r confinement; + * it owns no credential and has no journal. Provider and web traffic is + * brokered through the coordinator, preserving the hub-and-spoke map. + * SYSTEM and PROMPT are chunked and START carries the fixed route. + */ + FUGU_IMSG_AGENT_START, /* coordinator -> tool-r: struct agent_start + model\0label */ + FUGU_IMSG_AGENT_SYSTEM, /* coordinator -> tool-r: system/context chunk */ + FUGU_IMSG_AGENT_PROMPT, /* coordinator -> tool-r: task prompt chunk */ + FUGU_IMSG_AGENT_RUN, /* coordinator -> tool-r: start ephemeral Turn */ + FUGU_IMSG_AGENT_REQUEST, /* tool-r -> coordinator: provider JSON chunk */ + FUGU_IMSG_AGENT_REQUEST_END, /* tool-r -> coordinator: request complete */ + FUGU_IMSG_AGENT_WEB_REQ, /* tool-r -> coordinator: struct tool_req */ + FUGU_IMSG_AGENT_WEB_ARG, /* tool-r -> coordinator: web input JSON chunk */ + FUGU_IMSG_AGENT_WEB_RUN, /* tool-r -> coordinator: execute web call */ + FUGU_IMSG_AGENT_WEB_OUT, /* coordinator -> tool-r: web result chunk */ + FUGU_IMSG_AGENT_WEB_RESULT, /* coordinator -> tool-r: terminal result */ + FUGU_IMSG_AGENT_USAGE, /* tool-r -> coordinator: provider usage */ + FUGU_IMSG_AGENT_PROGRESS, /* tool-r -> coordinator: bounded status text */ + FUGU_IMSG_AGENT_OUT, /* tool-r -> coordinator: final report chunk */ + FUGU_IMSG_AGENT_RESULT, /* tool-r -> coordinator: struct agent_result */ + /* teardown: coordinator -> worker */ FUGU_IMSG_SHUTDOWN, @@ -323,18 +355,29 @@ enum ui_state { /* * fugu-tool's HELLO: the two configuration gates that shape the - * kernel confinement (invariants.md I5/I6), followed by the protect - * globs as NUL-separated trailing bytes (protectlen = imsg length - * minus this struct). The working directory is inherited through - * fork, so it is not sent. + * kernel confinement (invariants.md I5/I6). Normal coordinator traffic + * inherits the pre-expanded protect snapshot as a regular-file descriptor; + * this keeps an arbitrarily large startup set out of the bounded imsg payload. + * The legacy trailing-glob form remains useful to focused worker harnesses. + * The working directory is inherited through fork, so it is not sent. */ struct hello_tool { int allow_write; int allow_subprocess_net; int max_bg_jobs; /* concurrent background-job cap */ - /* protect globs follow as NUL-separated bytes, if any */ + int agent; /* 1: ephemeral subagent orchestrator */ + int protect_snapshot; /* 1: inherited fd holds literal paths */ + uint64_t protect_len; /* exact bytes at FUGU_PROTECT_FD */ + /* With protect_snapshot 0, legacy protect globs may follow as + * NUL-separated bytes. With it set, no bytes follow. */ }; +/* REQUEST_END may select a provider per correlated request. An empty + * payload retains the active lead provider for backward-compatible tests. */ +struct api_request_end { + int provider_slot; /* 0: implicit default; 1..: named block */ +}; + /* * fugu-web's HELLO. The fixed header is followed, in order, by exactly * tokenlen Kagi-token bytes, allowlen http_allow bytes, and blocklen @@ -393,6 +436,22 @@ struct tool_result { int is_error; }; +/* Fixed subagent launch policy. The payload continues as model\0label, + * both NUL-terminated and bounded by the constants above. */ +struct agent_start { + int provider_type; /* PROVIDER_* used by the request builder */ + int provider_slot; /* endpoint/key slot in fugu-api */ + int web_search; /* offer web_search and web_fetch */ + int http_request; /* offer http_request */ + int64_t max_tokens; /* per-generation output ceiling */ +}; + +struct agent_result { + int is_error; /* the report describes a failed subagent */ + int truncated; /* the 25-round bound forced conclusion */ + int iterations; /* provider tool rounds consumed */ +}; + struct tool_turn_ack { uint32_t control; /* BEGIN, COMMIT, or ABORT */ }; @@ -423,15 +482,19 @@ struct hello_prov { }; /* One model id out of a provider's listing (MODEL, UI_MODEL); the - * provider name and the model id follow as "name\0id". */ + * provider name and the model id follow as "name\0id". A zero window + * means that the listing supplied no usable metadata. */ struct model_ent { - int slot; /* 0: the default provider; 1..: blocks */ + int slot; /* 0: the default provider; 1..: blocks */ + int64_t context_window; }; /* A picker selection (UI_SELECT_MODEL) or an activation order to the - * custodian (SET_PROVIDER); the model id follows as trailing bytes. */ + * custodian (SET_PROVIDER); the model id follows as trailing bytes for + * UI_SELECT_MODEL. */ struct model_sel { - int slot; + int slot; + int64_t context_window; }; /* The terminal frame of a compose exchange (EDIT_RESULT, UI_COMPOSED). blob - fdc59e764bba62e00f01e69460ccd458a9ae7fca blob + 8f27e41feb168bb31de52e29c0a3790e4662b198 --- src/fugu-api/Makefile +++ src/fugu-api/Makefile @@ -1,7 +1,7 @@ PROG= fugu-api SRCS= main.c worker.c imsgev.c \ http.c anthropic_stream.c openai_stream.c sse.c json.c buf.c \ - log.c xmalloc.c + model_window.c log.c xmalloc.c BINDIR= ${LIBEXECDIR}/fugu MAN= blob - 7cb3f086e4f9c2285f20432c16ca48e520aab32a blob + 0d4591c227b88282ec35a94f224dd3819ac44845 --- src/fugu-api/main.c +++ src/fugu-api/main.c @@ -26,9 +26,18 @@ */ #include +#include +#include +#include +#include +#include +#include #include #include +#include +#include +#include #include #include #include @@ -42,6 +51,7 @@ #include "xmalloc.h" #include "buf.h" #include "json.h" +#include "model_window.h" #include "http.h" #include "anthropic.h" #include "openai.h" @@ -54,6 +64,15 @@ #define API_ERRBODY_READ_MS 1000 #define API_ERRBODY_TOTAL_S 5 #define API_AUTH_HDR_MAX (FUGU_SECRET_MAX + 128) +#define API_REQUEST_MAX 65 /* lead plus one full tool-call wave */ +#define API_NETWORK_MAX 8 /* bounds concurrent hostile stream state */ +#define API_REQUEST_MEMORY_MAX (64 * 1024 * 1024) +#define API_IPC_HIGH_WATER 256 +#define API_IPC_LOW_WATER 128 +#define API_READ_BUDGET (64 * 1024) +#define API_RESUME_MS 10 +#define API_RESPONSE_WIRE_MAX (64 * 1024 * 1024) +#define API_RESPONSE_TOTAL_S (10 * 60) /* model listing (behavior.md 2.1): bounded against a hostile provider */ #define LIST_TIMEOUT 10000 @@ -119,21 +138,77 @@ struct secret_filter { int ready; }; +struct api; + +enum request_phase { + REQ_ASSEMBLY, + REQ_REJECTED, + REQ_QUEUED, + REQ_DNS, + REQ_CONNECT, + REQ_HANDSHAKE, + REQ_WRITE, + REQ_HEADERS, + REQ_BODY, + REQ_ERROR_BODY, + REQ_BACKOFF +}; + +struct api_request { + TAILQ_ENTRY(api_request) entry; + struct api *api; + struct worker *worker; + uint32_t id; + size_t slot; + enum request_phase phase; + struct buf body; + struct buf wire; + struct buf errbody; + size_t received; + size_t wire_off; + size_t body_off; + int attempt; + int terminal; + int retryable; + int retry_after; + size_t response_text; + size_t response_bytes; + size_t tool_args_total; + struct a_usage usage; + char lasterr[FUGU_ERROR_MAX]; + struct secret_stream text_stream; + struct tool_stream tool_streams[API_TOOL_STREAM_MAX]; + struct http http; + int http_live; + struct addrinfo *addrs; + struct addrinfo *addr; + struct event_asr *resolver; + struct event io_ev; + struct event timer_ev; + int io_added; + int timer_added; + int paused; + int network_slot; + struct timespec deadline; + struct timespec total_deadline; + struct anthropic_stream *anthropic; + struct openai_stream *openai; +}; + +TAILQ_HEAD(api_requests, api_request); + struct api { struct tls_config *tlscfg; struct prov *provs; size_t nprov; - size_t active; /* requests go here */ - struct buf body; /* accumulating request */ - int terminal; /* a DONE/ERROR was relayed */ - size_t response_text; /* current response, bounded */ - size_t tool_args_total; /* all calls in this response */ - struct a_usage usage; /* hostile totals, checked */ + size_t active; /* lead's selected provider */ int retry_ms; /* backoff base; test override */ - char lasterr[FUGU_ERROR_MAX]; /* last retryable failure */ struct secret_filter filter; - struct secret_stream text_stream; - struct tool_stream tool_streams[API_TOOL_STREAM_MAX]; + struct api_requests requests; + size_t nrequests; + size_t ninflight; + size_t request_memory; + int list_pending; }; static const char * @@ -226,6 +301,8 @@ api_lockdown(struct worker *w, struct imsg *imsg) strlcpy(v->port, h.port, sizeof(v->port)); strlcpy(v->path, h.path, sizeof(v->path)); v->key = copy_key(p + sizeof(h), len - sizeof(h)); + if (len > sizeof(h)) + explicit_bzero((u_char *)p + sizeof(h), len - sizeof(h)); /* Load the trust store now, while rpath still permits it. */ if ((a->tlscfg = tls_config_new()) == NULL) @@ -268,6 +345,8 @@ prov_add(struct api *a, struct imsg *imsg) strlcpy(v->port, hp.port, sizeof(v->port)); strlcpy(v->path, hp.path, sizeof(v->path)); v->key = copy_key(p + sizeof(hp), len - sizeof(hp)); + if (len > sizeof(hp)) + explicit_bzero((u_char *)p + sizeof(hp), len - sizeof(hp)); a->filter.ready = 0; } @@ -407,6 +486,29 @@ contains_secret(struct api *a, const void *data, size_ return (0); } +/* Match across untruncated provider fields without allocating a joined + * string. This closes the prefix-reflection case where snprintf truncation + * would otherwise remove the tail of a long configured credential. */ +static int +contains_secret_parts(struct api *a, const char *const *parts, size_t nparts) +{ + struct secret_filter *f = &a->filter; + size_t i, j; + int state = 0; + + secret_filter_build(a); + if (f->maxlen == 0) + return (0); + for (i = 0; i < nparts; i++) { + if (parts[i] == NULL) + continue; + for (j = 0; parts[i][j] != '\0'; j++) + if (secret_step(f, &state, (u_char)parts[i][j])) + return (1); + } + return (0); +} + static void secret_stream_clear(struct secret_stream *s) { @@ -416,15 +518,15 @@ secret_stream_clear(struct secret_stream *s) } static void -secret_streams_clear(struct api *a) +secret_streams_clear(struct api_request *r) { int i; - secret_stream_clear(&a->text_stream); + secret_stream_clear(&r->text_stream); for (i = 0; i < API_TOOL_STREAM_MAX; i++) { - if (a->tool_streams[i].used) - secret_stream_clear(&a->tool_streams[i].stream); - memset(&a->tool_streams[i], 0, sizeof(a->tool_streams[i])); + if (r->tool_streams[i].used) + secret_stream_clear(&r->tool_streams[i].stream); + memset(&r->tool_streams[i], 0, sizeof(r->tool_streams[i])); } } @@ -462,47 +564,52 @@ secret_stream_feed(struct api *a, struct secret_stream } static struct tool_stream * -tool_stream_find(struct api *a, int index) +tool_stream_find(struct api_request *r, int index) { int i; for (i = 0; i < API_TOOL_STREAM_MAX; i++) - if (a->tool_streams[i].used && - a->tool_streams[i].index == index) - return (&a->tool_streams[i]); + if (r->tool_streams[i].used && + r->tool_streams[i].index == index) + return (&r->tool_streams[i]); return (NULL); } static struct tool_stream * -tool_stream_begin(struct api *a, int index) +tool_stream_begin(struct api_request *r, int index) { int i; - if (index < 0 || tool_stream_find(a, index) != NULL) + if (index < 0 || tool_stream_find(r, index) != NULL) return (NULL); for (i = 0; i < API_TOOL_STREAM_MAX; i++) { - if (a->tool_streams[i].used) + if (r->tool_streams[i].used) continue; - a->tool_streams[i].used = 1; - a->tool_streams[i].index = index; - buf_init(&a->tool_streams[i].stream.pending); - return (&a->tool_streams[i]); + r->tool_streams[i].used = 1; + r->tool_streams[i].index = index; + buf_init(&r->tool_streams[i].stream.pending); + return (&r->tool_streams[i]); } return (NULL); } static void -relay(struct worker *w, uint32_t type, const void *data, size_t len) +relay_id(struct worker *w, uint32_t id, uint32_t type, const void *data, + size_t len) { - if (imsg_compose(&w->iev.ibuf, type, 0, -1, -1, data, len) == -1) - fatal("imsg_compose"); - if (imsgbuf_flush(&w->iev.ibuf) == -1) - fatal("imsgbuf_flush"); /* stream events out immediately */ + if (imsgev_compose(&w->iev, type, id, -1, -1, data, len) == -1) + fatal("compose provider event"); } static void -relay_bytes(struct worker *w, uint32_t type, const void *data, size_t len) +relay(struct api_request *r, uint32_t type, const void *data, size_t len) { + relay_id(r->worker, r->id, type, data, len); +} + +static void +relay_bytes(struct api_request *r, uint32_t type, const void *data, size_t len) +{ const u_char *p = data; size_t off, n; @@ -510,12 +617,12 @@ relay_bytes(struct worker *w, uint32_t type, const voi n = len - off; if (n > FUGU_CHUNK) n = FUGU_CHUNK; - relay(w, type, p + off, n); + relay(r, type, p + off, n); } } static void -relay_tool_bytes(struct worker *w, int index, const void *data, size_t len) +relay_tool_bytes(struct api_request *r, int index, const void *data, size_t len) { const u_char *p = data; size_t off, n; @@ -529,53 +636,51 @@ relay_tool_bytes(struct worker *w, int index, const vo buf_init(&b); buf_add(&b, &index, sizeof(index)); buf_add(&b, p + off, n); - relay(w, FUGU_IMSG_A_TOOL_INPUT, b.data, b.len); + relay(r, FUGU_IMSG_A_TOOL_INPUT, b.data, b.len); buf_free(&b); } } static int -filter_text(struct worker *w, const void *data, size_t len, int flush) +filter_text(struct api_request *req, const void *data, size_t len, int flush) { - struct api *a = w->ctx; struct buf out; - int r; + int rc; buf_init(&out); - r = secret_stream_feed(a, &a->text_stream, data, len, flush, &out); - if (r == 0) - relay_bytes(w, FUGU_IMSG_A_TEXT, out.data, out.len); + rc = secret_stream_feed(req->api, &req->text_stream, data, len, flush, + &out); + if (rc == 0) + relay_bytes(req, FUGU_IMSG_A_TEXT, out.data, out.len); buf_freezero(&out); - return (r); + return (rc); } static int -filter_tool(struct worker *w, struct tool_stream *ts, const void *data, +filter_tool(struct api_request *r, struct tool_stream *ts, const void *data, size_t len, int flush) { - struct api *a = w->ctx; struct buf out; - int r; + int rc; buf_init(&out); - r = secret_stream_feed(a, &ts->stream, data, len, flush, &out); - if (r == 0) - relay_tool_bytes(w, ts->index, out.data, out.len); + rc = secret_stream_feed(r->api, &ts->stream, data, len, flush, &out); + if (rc == 0) + relay_tool_bytes(r, ts->index, out.data, out.len); buf_freezero(&out); - return (r); + return (rc); } static int -filters_flush(struct worker *w) +filters_flush(struct api_request *r) { - struct api *a = w->ctx; int i; - if (filter_text(w, NULL, 0, 1) == -1) + if (filter_text(r, NULL, 0, 1) == -1) return (-1); for (i = 0; i < API_TOOL_STREAM_MAX; i++) - if (a->tool_streams[i].used && - filter_tool(w, &a->tool_streams[i], NULL, 0, 1) == -1) + if (r->tool_streams[i].used && + filter_tool(r, &r->tool_streams[i], NULL, 0, 1) == -1) return (-1); return (0); } @@ -597,42 +702,42 @@ usage_add(struct a_usage *dst, const struct a_usage *s return (0); } -static void relay_error(struct worker *, const char *); +static void relay_error(struct api_request *, const char *); static void on_event(const struct astream_event *ev, void *arg) { - struct worker *w = arg; - struct api *a = w->ctx; + struct api_request *r = arg; + struct api *a = r->api; /* * The stream is hostile (T3): exactly one terminal per request * reaches the coordinator. A single feed can decode events past * a terminal, so drop them here, not just in the read loop. */ - if (a->terminal) + if (r->terminal) return; switch (ev->kind) { case ASTREAM_TEXT: { size_t off; - if (a->response_text > FUGU_RESPONSE_MAX || - ev->textlen > FUGU_RESPONSE_MAX - a->response_text) { + if (r->response_text > FUGU_RESPONSE_MAX || + ev->textlen > FUGU_RESPONSE_MAX - r->response_text) { static const char msg[] = "provider response exceeds 16 MiB"; - relay_error(w, msg); + relay_error(r, msg); break; } - a->response_text += ev->textlen; + r->response_text += ev->textlen; for (off = 0; off < ev->textlen; ) { size_t n = ev->textlen - off; if (n > FUGU_CHUNK) n = FUGU_CHUNK; - if (filter_text(w, ev->text + off, n, 0) == -1) { - relay_error(w, + if (filter_text(r, ev->text + off, n, 0) == -1) { + relay_error(r, "provider response contained an API credential"); break; } @@ -652,11 +757,11 @@ on_event(const struct astream_event *ev, void *arg) namelen >= FUGU_TOOL_NAME_MAX || contains_secret(a, ev->tool_id, idlen) || contains_secret(a, ev->tool_name, namelen)) { - relay_error(w, "invalid provider tool identifier"); + relay_error(r, "invalid provider tool identifier"); break; } - if (tool_stream_begin(a, ev->index) == NULL) { - relay_error(w, "invalid provider tool-call sequence"); + if (tool_stream_begin(r, ev->index) == NULL) { + relay_error(r, "invalid provider tool-call sequence"); break; } @@ -667,7 +772,7 @@ on_event(const struct astream_event *ev, void *arg) buf_addstr(&b, ev->tool_id); buf_addc(&b, '\0'); buf_addstr(&b, ev->tool_name); - relay(w, FUGU_IMSG_A_TOOL_BEGIN, b.data, b.len); + relay(r, FUGU_IMSG_A_TOOL_BEGIN, b.data, b.len); buf_free(&b); break; } @@ -675,24 +780,24 @@ on_event(const struct astream_event *ev, void *arg) struct tool_stream *ts; size_t off; - if ((ts = tool_stream_find(a, ev->index)) == NULL || + if ((ts = tool_stream_find(r, ev->index)) == NULL || (ev->partial == NULL && ev->partiallen != 0) || ev->partiallen > FUGU_TOOL_ARG_MAX - ts->stream.total || ev->partiallen > - FUGU_TOOL_ARGS_TOTAL_MAX - a->tool_args_total) { - relay_error(w, "invalid provider tool input"); + FUGU_TOOL_ARGS_TOTAL_MAX - r->tool_args_total) { + relay_error(r, "invalid provider tool input"); break; } ts->stream.total += ev->partiallen; - a->tool_args_total += ev->partiallen; + r->tool_args_total += ev->partiallen; for (off = 0; off < ev->partiallen; ) { size_t n = ev->partiallen - off; if (n > FUGU_CHUNK) n = FUGU_CHUNK; - if (filter_tool(w, ts, ev->partial + off, n, 0) == -1) { - relay_error(w, + if (filter_tool(r, ts, ev->partial + off, n, 0) == -1) { + relay_error(r, "provider response contained an API credential"); break; } @@ -707,13 +812,13 @@ on_event(const struct astream_event *ev, void *arg) u.output_tokens = ev->output_tokens; u.cache_read = ev->cache_read; u.cache_write = ev->cache_write; - if (usage_add(&a->usage, &u) == -1) { + if (usage_add(&r->usage, &u) == -1) { static const char msg[] = "invalid provider token usage"; - relay_error(w, msg); + relay_error(r, msg); break; } - relay(w, FUGU_IMSG_A_USAGE, &u, sizeof(u)); + relay(r, FUGU_IMSG_A_USAGE, &u, sizeof(u)); break; } case ASTREAM_DONE: { @@ -723,39 +828,45 @@ on_event(const struct astream_event *ev, void *arg) (len = strlen(ev->stop_reason)) == 0 || len >= FUGU_STOP_REASON_MAX || contains_secret(a, ev->stop_reason, len)) { - relay_error(w, "invalid provider stop reason"); + relay_error(r, "invalid provider stop reason"); break; } - if (filters_flush(w) == -1) { - relay_error(w, + if (filters_flush(r) == -1) { + relay_error(r, "provider response contained an API credential"); break; } - relay(w, FUGU_IMSG_A_DONE, ev->stop_reason, len); - a->terminal = 1; - secret_streams_clear(a); + relay(r, FUGU_IMSG_A_DONE, ev->stop_reason, len); + r->terminal = 1; + secret_streams_clear(r); break; } case ASTREAM_ERROR: /* Preserve both provider fields when available (section 8). */ { char reason[FUGU_ERROR_MAX]; + const char *parts[3]; - (void)snprintf(reason, sizeof(reason), "%s%s%s", - ev->err_type != NULL ? ev->err_type : "error", - ev->err_message != NULL && ev->err_message[0] != '\0' ? - ": " : "", - ev->err_message != NULL ? ev->err_message : ""); - relay_error(w, reason); + parts[0] = ev->err_type != NULL ? ev->err_type : "error"; + parts[1] = ev->err_message != NULL && + ev->err_message[0] != '\0' ? ": " : ""; + parts[2] = ev->err_message != NULL ? ev->err_message : ""; + if (contains_secret_parts(a, parts, 3)) + relay_error(r, "\n"); + else { + (void)snprintf(reason, sizeof(reason), "%s%s%s", + parts[0], parts[1], parts[2]); + relay_error(r, reason); + } } break; } } static void -relay_error(struct worker *w, const char *msg) +relay_error(struct api_request *r, const char *msg) { - struct api *a = w->ctx; + struct api *a = r->api; /* CR/LF are forbidden in configured credentials. A single newline is * therefore a fail-closed replacement even for a one-byte key; fixed * prose cannot make that guarantee. */ @@ -766,105 +877,27 @@ relay_error(struct worker *w, const char *msg) if (contains_secret(a, msg, len)) { msg = reflected; len = 1; - } - secret_streams_clear(a); - relay(w, FUGU_IMSG_A_ERROR, msg, len); - a->terminal = 1; + } else if (len >= FUGU_ERROR_MAX) + len = FUGU_ERROR_MAX - 1; + secret_streams_clear(r); + relay(r, FUGU_IMSG_A_ERROR, msg, len); + r->terminal = 1; } -/* The provider's own message out of a non-2xx body, or the status. */ -static void -error_reason(struct http *h, const struct prov *v, char *out, size_t outsz) -{ - struct buf b; - struct timespec t0, now; - char tmp[4096]; - ssize_t n; - size_t want; - buf_init(&b); - clock_gettime(CLOCK_MONOTONIC, &t0); - while (b.len < API_ERRBODY_MAX) { - want = API_ERRBODY_MAX - b.len; - if (want > sizeof(tmp)) - want = sizeof(tmp); - n = http_body(h, tmp, want, API_ERRBODY_READ_MS); - if (n <= 0) - break; - buf_add(&b, tmp, (size_t)n); - clock_gettime(CLOCK_MONOTONIC, &now); - if (now.tv_sec - t0.tv_sec >= API_ERRBODY_TOTAL_S) - break; - } - if ((v->type == PROVIDER_OPENAI ? - openai_error_body(b.data, b.len, out, outsz) : - anthropic_error_body(b.data, b.len, out, outsz)) == -1) - (void)snprintf(out, outsz, "HTTP %d from provider", - h->status); - buf_free(&b); -} +/* + * Provider requests are independent libevent state machines. The API role + * remains the sole credential custodian, but a slow subagent connection no + * longer stalls the coordinator channel or another member of the wave. + */ +static void request_start_attempt(struct api_request *); +static void request_io(int, short, void *); +static void request_timer(int, short, void *); +static void request_read(struct api_request *); +static void list_all(struct worker *, struct api *); -static void -stream_anthropic(struct worker *w, struct http *h) -{ - struct api *a = w->ctx; - struct anthropic_stream *stream; - char rbuf[8192]; - ssize_t n; - - stream = anthropic_stream_new(on_event, w); - for (;;) { - n = http_body(h, rbuf, sizeof(rbuf), API_TIMEOUT); - if (n < 0) { - if (!a->terminal) - relay_error(w, http_error(h)); - break; - } - if (n == 0) - break; - if (anthropic_stream_feed(stream, rbuf, (size_t)n) == -1) { - if (!a->terminal) - relay_error(w, anthropic_stream_error(stream)); - break; - } - if (a->terminal) - break; - } - anthropic_stream_free(stream); -} - -static void -stream_openai(struct worker *w, struct http *h) -{ - struct api *a = w->ctx; - struct openai_stream *stream; - char rbuf[8192]; - ssize_t n; - - stream = openai_stream_new(on_event, w); - for (;;) { - n = http_body(h, rbuf, sizeof(rbuf), API_TIMEOUT); - if (n < 0) { - if (!a->terminal) - relay_error(w, http_error(h)); - break; - } - if (n == 0) - break; - if (openai_stream_feed(stream, rbuf, (size_t)n) == -1) { - if (!a->terminal) - relay_error(w, openai_stream_error(stream)); - break; - } - if (a->terminal) - break; - } - openai_stream_free(stream); -} - -/* Retry-After delta-seconds, untrusted (I13), clamped to [1, 60]. */ static int -retry_after(struct http *h) +request_retry_after(struct http *h) { const char *val, *errstr; size_t vlen; @@ -878,7 +911,7 @@ retry_after(struct http *h) num[vlen] = '\0'; v = strtonum(num, 0, 1000000, &errstr); if (errstr != NULL) - return (-1); /* HTTP-date or garbage: use backoff */ + return (-1); if (v < 1) v = 1; if (v > 60) @@ -886,91 +919,12 @@ retry_after(struct http *h) return ((int)v); } -/* - * One attempt. Returns 0 when a terminal was relayed (the turn is - * decided), -1 when the failure is retryable under the pinned policy - * (behavior.md section 8): connect or TLS handshake failure, HTTP - * 429, or any 5xx -- all before the first content delta. The reason - * lands in a->lasterr; *ra carries a clamped Retry-After or -1. - */ static int -api_once(struct worker *w, const char *headers, int *ra) +request_auth_headers(const struct prov *v, char *out, size_t outsz) { - struct api *a = w->ctx; - struct http h; + int n; - struct prov *v = &a->provs[a->active]; - - *ra = -1; - if (http_connect(&h, v->host, v->port, v->host, a->tlscfg, - API_TIMEOUT, NULL, NULL) == -1) { - strlcpy(a->lasterr, http_error(&h), sizeof(a->lasterr)); - http_close(&h); - return (-1); - } - if (http_request(&h, "POST", v->path, headers, - a->body.data, a->body.len, API_TIMEOUT) == -1 || - http_response(&h, API_TIMEOUT) == -1) { - relay_error(w, http_error(&h)); - http_close(&h); - return (0); - } - if (h.status == 429 || h.status >= 500) { - *ra = retry_after(&h); - error_reason(&h, v, a->lasterr, sizeof(a->lasterr)); - http_close(&h); - return (-1); - } - if (h.status < 200 || h.status > 299) { - /* any other 4xx ends the turn, with the provider's own - * message and type where available */ - char reason[FUGU_ERROR_MAX]; - - error_reason(&h, v, reason, sizeof(reason)); - relay_error(w, reason); - http_close(&h); - return (0); - } - if (v->type == PROVIDER_OPENAI) - stream_openai(w, &h); - else - stream_anthropic(w, &h); - http_close(&h); - - /* the provider always ends with stop_reason; if the body ended - * without one and without an error, say so rather than hang */ - if (!a->terminal) - relay_error(w, "stream ended without a stop reason"); - return (0); -} - -/* Exponential backoff plus jitter (1/2/4/8/16 s, each +/- up to - * half), or the endpoint's clamped Retry-After when it sent one. */ -static void -backoff(struct api *a, int attempt, int ra) -{ - struct timespec ts; - unsigned int ms; - - if (ra > 0) - ms = (unsigned int)ra * 1000; - else { - ms = (unsigned int)a->retry_ms << (attempt - 1); - ms = ms / 2 + arc4random_uniform(ms + 1); - } - ts.tv_sec = ms / 1000; - ts.tv_nsec = (long)(ms % 1000) * 1000000; - (void)nanosleep(&ts, NULL); -} - -/* The provider's auth headers; the key never leaves this process. */ -static int -auth_headers(const struct prov *v, char *out, size_t outsz) -{ - int n; - - if (v->type == PROVIDER_OPENAI) n = snprintf(out, outsz, "Authorization: Bearer %s\r\n" "content-type: application/json\r\n", v->key); @@ -982,53 +936,766 @@ auth_headers(const struct prov *v, char *out, size_t o return (n < 0 || (size_t)n >= outsz ? -1 : 0); } -/* Run the accumulated request against the endpoint and stream back. */ +static struct api_request * +request_find(struct api *a, uint32_t id) +{ + struct api_request *r; + + TAILQ_FOREACH(r, &a->requests, entry) + if (r->id == id) + return (r); + return (NULL); +} + static void -api_run(struct worker *w) +request_deadline_in(struct api_request *r, int ms) { - struct api *a = w->ctx; - char headers[API_AUTH_HDR_MAX]; + clock_gettime(CLOCK_MONOTONIC, &r->deadline); + r->deadline.tv_sec += ms / 1000; + r->deadline.tv_nsec += (long)(ms % 1000) * 1000000L; + if (r->deadline.tv_nsec >= 1000000000L) { + r->deadline.tv_sec++; + r->deadline.tv_nsec -= 1000000000L; + } +} + +static void +request_total_deadline_in(struct api_request *r, int ms) +{ + clock_gettime(CLOCK_MONOTONIC, &r->total_deadline); + r->total_deadline.tv_sec += ms / 1000; + r->total_deadline.tv_nsec += (long)(ms % 1000) * 1000000L; + if (r->total_deadline.tv_nsec >= 1000000000L) { + r->total_deadline.tv_sec++; + r->total_deadline.tv_nsec -= 1000000000L; + } +} + +static int +request_deadline_left(const struct api_request *r) +{ + struct timespec now, d; + long long ms; + + clock_gettime(CLOCK_MONOTONIC, &now); + if (timespeccmp(&now, &r->deadline, >=)) + return (0); + timespecsub(&r->deadline, &now, &d); + ms = (long long)d.tv_sec * 1000 + d.tv_nsec / 1000000; + if (ms < 1) + return (1); + if (ms > INT_MAX) + return (INT_MAX); + return ((int)ms); +} + +static int +request_total_deadline_left(const struct api_request *r) +{ + struct timespec now, d; + long long ms; + + clock_gettime(CLOCK_MONOTONIC, &now); + if (timespeccmp(&now, &r->total_deadline, >=)) + return (0); + timespecsub(&r->total_deadline, &now, &d); + ms = (long long)d.tv_sec * 1000 + d.tv_nsec / 1000000; + if (ms < 1) + return (1); + if (ms > INT_MAX) + return (INT_MAX); + return ((int)ms); +} + +static void +request_timer_clear(struct api_request *r) +{ + if (r->timer_added) { + evtimer_del(&r->timer_ev); + r->timer_added = 0; + } +} + +static void +request_io_clear(struct api_request *r) +{ + if (r->io_added) { + event_del(&r->io_ev); + r->io_added = 0; + } +} + +static void +request_attempt_clear(struct api_request *r) +{ + request_io_clear(r); + request_timer_clear(r); + if (r->resolver != NULL) { + event_asr_abort(r->resolver); + r->resolver = NULL; + } + if (r->anthropic != NULL) { + anthropic_stream_free(r->anthropic); + r->anthropic = NULL; + } + if (r->openai != NULL) { + openai_stream_free(r->openai); + r->openai = NULL; + } + if (r->http_live) { + http_close(&r->http); + r->http_live = 0; + } + if (r->addrs != NULL) { + freeaddrinfo(r->addrs); + r->addrs = NULL; + r->addr = NULL; + } + buf_freezero(&r->wire); + buf_init(&r->wire); + buf_freezero(&r->errbody); + buf_init(&r->errbody); + r->wire_off = 0; + r->body_off = 0; + r->paused = 0; +} + +static void +request_activate_queued(struct api *a) +{ + struct api_request *r; + + TAILQ_FOREACH(r, &a->requests, entry) { + if (a->ninflight >= API_NETWORK_MAX) + break; + if (r->phase != REQ_QUEUED) + continue; + a->ninflight++; + r->network_slot = 1; + request_start_attempt(r); + } +} + +static void +request_release_network(struct api_request *r) +{ + struct api *a = r->api; + + if (!r->network_slot) + return; + if (a->ninflight == 0) + fatalx("fugu-api: inflight accounting underflow"); + r->network_slot = 0; + a->ninflight--; + request_activate_queued(a); +} + +static void +request_free(struct api_request *r) +{ + struct api *a = r->api; + struct worker *w = r->worker; + int list_pending; + + request_attempt_clear(r); + secret_streams_clear(r); + TAILQ_REMOVE(&a->requests, r, entry); + if (a->nrequests == 0) + fatalx("fugu-api: request accounting underflow"); + a->nrequests--; + request_release_network(r); + if (r->body.cap > a->request_memory) + fatalx("fugu-api: request memory accounting underflow"); + a->request_memory -= r->body.cap; + buf_free(&r->body); + buf_freezero(&r->wire); + buf_freezero(&r->errbody); + list_pending = a->nrequests == 0 && a->list_pending; + if (list_pending) + a->list_pending = 0; + free(r); + if (list_pending) + list_all(w, a); +} + +static void +request_schedule_io(struct api_request *r, short events, int timeout_ms) +{ + struct timeval tv; + int left; + + request_io_clear(r); + left = request_deadline_left(r); + if (r->phase == REQ_BODY) { + int total = request_total_deadline_left(r); + + if (total < left) + left = total; + } + if (left < timeout_ms) + timeout_ms = left; + if (timeout_ms < 1) + timeout_ms = 1; + tv.tv_sec = timeout_ms / 1000; + tv.tv_usec = (timeout_ms % 1000) * 1000; + event_set(&r->io_ev, r->http.fd, events, request_io, r); + if (event_add(&r->io_ev, &tv) == -1) + fatal("event_add provider request"); + r->io_added = 1; +} + +static void +request_schedule_timer(struct api_request *r, unsigned int ms) +{ + struct timeval tv; + + request_timer_clear(r); + tv.tv_sec = ms / 1000; + tv.tv_usec = (ms % 1000) * 1000; + evtimer_set(&r->timer_ev, request_timer, r); + if (evtimer_add(&r->timer_ev, &tv) == -1) + fatal("evtimer_add provider request"); + r->timer_added = 1; +} + +static void +request_fail(struct api_request *r, const char *reason) +{ + if (!r->terminal) + relay_error(r, reason); + request_free(r); +} + +/* Retryable failures are only connect/TLS, HTTP 429, and 5xx before a + * content delta. Each request owns its timer, so retries remain concurrent. */ +static void +request_retry(struct api_request *r, const char *reason, int ra) +{ char notice[1200]; - int attempt, ra, len; + unsigned int ms; + int len; - secret_streams_clear(a); - a->terminal = 0; - a->response_text = 0; - a->tool_args_total = 0; - memset(&a->usage, 0, sizeof(a->usage)); - - if (a->provs[a->active].key == NULL) { - relay_error(w, "no API key configured"); - goto reset; + if (reason != r->lasterr) { + if (contains_secret(r->api, reason, strlen(reason))) + strlcpy(r->lasterr, "\n", sizeof(r->lasterr)); + else + strlcpy(r->lasterr, reason, sizeof(r->lasterr)); } - if (auth_headers(&a->provs[a->active], headers, sizeof(headers)) == -1) { - relay_error(w, "API key is too long"); - goto reset; + request_attempt_clear(r); + request_release_network(r); + if (r->attempt >= API_TRIES) { + relay_error(r, r->lasterr); + request_free(r); + return; } + len = snprintf(notice, sizeof(notice), "retry %d/%d: %s", + r->attempt + 1, API_TRIES, r->lasterr); + if (len < 0 || len >= (int)sizeof(notice)) + len = (int)sizeof(notice) - 1; + if (contains_secret(r->api, notice, (size_t)len)) + relay(r, FUGU_IMSG_A_RETRY, "\n", 1); + else + relay(r, FUGU_IMSG_A_RETRY, notice, (size_t)len); + if (ra > 0) + ms = (unsigned int)ra * 1000; + else { + ms = (unsigned int)r->api->retry_ms << (r->attempt - 1); + ms = ms / 2 + arc4random_uniform(ms + 1); + } + r->phase = REQ_BACKOFF; + request_schedule_timer(r, ms); +} - for (attempt = 1; attempt <= API_TRIES; attempt++) { - if (api_once(w, headers, &ra) == 0) - goto reset; - if (attempt == API_TRIES) - break; - /* one retry notice per attempt (behavior.md section 8) */ - len = snprintf(notice, sizeof(notice), "retry %d/%d: %s", - attempt + 1, API_TRIES, a->lasterr); - if (len < 0 || len >= (int)sizeof(notice)) - len = (int)sizeof(notice) - 1; - if (contains_secret(a, notice, (size_t)len)) - relay(w, FUGU_IMSG_A_RETRY, "\n", 1); +static void +request_error_done(struct api_request *r) +{ + struct prov *v = &r->api->provs[r->slot]; + char *reason; + size_t reasonsz = API_ERRBODY_MAX + 32; + int retryable = r->retryable; + int retry_after = r->retry_after; + + reason = xcalloc(reasonsz, 1); + if ((v->type == PROVIDER_OPENAI ? + openai_error_body(r->errbody.data, r->errbody.len, reason, + reasonsz) : + anthropic_error_body(r->errbody.data, r->errbody.len, reason, + reasonsz)) == -1) + (void)snprintf(reason, reasonsz, "HTTP %d from provider", + r->http.status); + if (retryable) + request_retry(r, reason, retry_after); + else + request_fail(r, reason); + freezero(reason, reasonsz); +} + +/* Consume complete HTTP units already buffered. One means the request was + * freed; zero means the TLS socket must provide more bytes. */ +static int +request_process(struct api_request *r, int eof) +{ + char tmp[8192]; + ssize_t n; + int rc; + + for (;;) { + if (r->phase == REQ_HEADERS) { + rc = http_response_parse(&r->http, eof); + if (rc == -1) { + request_fail(r, http_error(&r->http)); + return (1); + } + if (rc == 0) + return (0); + if (r->http.status == 429 || r->http.status >= 500) { + r->retryable = 1; + r->retry_after = request_retry_after(&r->http); + r->phase = REQ_ERROR_BODY; + request_deadline_in(r, API_ERRBODY_TOTAL_S * 1000); + } else if (r->http.status < 200 || r->http.status > 299) { + r->retryable = 0; + r->retry_after = -1; + r->phase = REQ_ERROR_BODY; + request_deadline_in(r, API_ERRBODY_TOTAL_S * 1000); + } else { + if (r->api->provs[r->slot].type == PROVIDER_OPENAI) + r->openai = openai_stream_new(on_event, r); + else + r->anthropic = anthropic_stream_new(on_event, r); + r->phase = REQ_BODY; + request_deadline_in(r, API_TIMEOUT); + request_total_deadline_in(r, + API_RESPONSE_TOTAL_S * 1000); + } + } + + n = http_body_parse(&r->http, tmp, sizeof(tmp), eof); + if (n == HTTP_BODY_MORE) + return (0); + if (n == -1) { + if (r->phase == REQ_ERROR_BODY) + request_error_done(r); + else + request_fail(r, http_error(&r->http)); + return (1); + } + if (n == 0) { + if (r->phase == REQ_ERROR_BODY) + request_error_done(r); + else { + if (!r->terminal) + relay_error(r, + "stream ended without a stop reason"); + request_free(r); + } + return (1); + } + if (r->phase == REQ_ERROR_BODY) { + size_t keep = (size_t)n; + + if (keep > API_ERRBODY_MAX - r->errbody.len) + keep = API_ERRBODY_MAX - r->errbody.len; + if (keep > 0) + buf_add(&r->errbody, tmp, keep); + if (r->errbody.len == API_ERRBODY_MAX) { + request_error_done(r); + return (1); + } + continue; + } + /* Successful body data starts a fresh inactivity window, matching + * the old blocking http_body deadline without letting TLS WANT + * cycles extend it. */ + if ((size_t)n > API_RESPONSE_WIRE_MAX - r->response_bytes) { + request_fail(r, "provider response wire bound exceeded"); + return (1); + } + r->response_bytes += (size_t)n; + request_deadline_in(r, API_TIMEOUT); + if (r->openai != NULL) { + if (openai_stream_feed(r->openai, tmp, (size_t)n) == -1 && + !r->terminal) + relay_error(r, openai_stream_error(r->openai)); + } else if (anthropic_stream_feed(r->anthropic, tmp, + (size_t)n) == -1 && !r->terminal) + relay_error(r, anthropic_stream_error(r->anthropic)); + if (r->terminal) { + request_free(r); + return (1); + } + } +} + +static int +request_output_high(const struct api_request *r) +{ + return (imsgbuf_queuelen(&r->worker->iev.ibuf) >= + API_IPC_HIGH_WATER); +} + +static void +request_pause(struct api_request *r) +{ + request_io_clear(r); + r->paused = 1; + request_schedule_timer(r, API_RESUME_MS); +} + +static void +request_read(struct api_request *r) +{ + u_char tmp[8192]; + ssize_t n; + size_t total = 0; + + if (request_output_high(r)) { + request_pause(r); + return; + } + if (request_process(r, 0)) + return; + if (request_output_high(r)) { + request_pause(r); + return; + } + for (;;) { + n = tls_read(r->http.tls, tmp, sizeof(tmp)); + if (n > 0) { + total += (size_t)n; + buf_add(&r->http.in, tmp, (size_t)n); + if (request_process(r, 0)) + return; + if (request_output_high(r) || total >= API_READ_BUDGET) { + request_pause(r); + return; + } + continue; + } + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) { + request_schedule_io(r, n == TLS_WANT_POLLIN ? EV_READ : + EV_WRITE, r->phase == REQ_ERROR_BODY ? + API_ERRBODY_READ_MS : API_TIMEOUT); + return; + } + if (n == 0) { + (void)request_process(r, 1); + return; + } + if (r->phase == REQ_ERROR_BODY) + request_error_done(r); else - relay(w, FUGU_IMSG_A_RETRY, notice, (size_t)len); - backoff(a, attempt, ra); + request_fail(r, tls_error(r->http.tls) != NULL ? + tls_error(r->http.tls) : "provider connection lost"); + return; } - /* exhausted retries surface the final provider error */ - relay_error(w, a->lasterr); -reset: - secret_streams_clear(a); - buf_reset(&a->body); +} + +static void +request_write(struct api_request *r) +{ + ssize_t n; + + while (r->wire_off < r->wire.len) { + n = tls_write(r->http.tls, r->wire.data + r->wire_off, + r->wire.len - r->wire_off); + if (n > 0) { + r->wire_off += (size_t)n; + continue; + } + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) { + request_schedule_io(r, n == TLS_WANT_POLLIN ? EV_READ : + EV_WRITE, API_TIMEOUT); + return; + } + request_fail(r, tls_error(r->http.tls) != NULL ? + tls_error(r->http.tls) : "provider write failed"); + return; + } + buf_freezero(&r->wire); + buf_init(&r->wire); + r->wire_off = 0; + while (r->body_off < r->body.len) { + n = tls_write(r->http.tls, r->body.data + r->body_off, + r->body.len - r->body_off); + if (n > 0) { + r->body_off += (size_t)n; + continue; + } + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) { + request_schedule_io(r, n == TLS_WANT_POLLIN ? EV_READ : + EV_WRITE, API_TIMEOUT); + return; + } + request_fail(r, tls_error(r->http.tls) != NULL ? + tls_error(r->http.tls) : "provider write failed"); + return; + } + r->body_off = 0; + r->phase = REQ_HEADERS; + request_deadline_in(r, API_TIMEOUT); + request_read(r); } +static void +request_handshake(struct api_request *r) +{ + struct prov *v = &r->api->provs[r->slot]; + char headers[API_AUTH_HDR_MAX]; + int rc, buildrc; + + rc = tls_handshake(r->http.tls); + if (rc == TLS_WANT_POLLIN || rc == TLS_WANT_POLLOUT) { + request_schedule_io(r, rc == TLS_WANT_POLLIN ? EV_READ : EV_WRITE, + API_TIMEOUT); + return; + } + if (rc == -1) { + request_retry(r, tls_error(r->http.tls) != NULL ? + tls_error(r->http.tls) : "TLS handshake failed", -1); + return; + } + buildrc = request_auth_headers(v, headers, sizeof(headers)); + if (buildrc == 0) + buildrc = http_request_head_build(&r->http, &r->wire, "POST", 4, + v->path, strlen(v->path), headers, strlen(headers), + r->body.len); + explicit_bzero(headers, sizeof(headers)); + if (buildrc == -1) { + request_fail(r, "cannot build provider request"); + return; + } + r->phase = REQ_WRITE; + request_deadline_in(r, API_TIMEOUT); + request_write(r); +} + +static void +request_tls_start(struct api_request *r) +{ + struct prov *v = &r->api->provs[r->slot]; + + if ((r->http.tls = tls_client()) == NULL || + tls_configure(r->http.tls, r->api->tlscfg) == -1 || + tls_connect_socket(r->http.tls, r->http.fd, v->host) == -1) { + request_retry(r, r->http.tls != NULL && + tls_error(r->http.tls) != NULL ? tls_error(r->http.tls) : + "TLS setup failed", -1); + return; + } + r->phase = REQ_HANDSHAKE; + request_handshake(r); +} + +static void +request_connect_next(struct api_request *r) +{ + struct prov *v = &r->api->provs[r->slot]; + int fd, rc, saved = 0; + + while (r->addr != NULL) { + fd = socket(r->addr->ai_family, + r->addr->ai_socktype | SOCK_NONBLOCK, r->addr->ai_protocol); + if (fd == -1) { + saved = errno; + r->addr = r->addr->ai_next; + continue; + } + r->http.fd = fd; + rc = connect(fd, r->addr->ai_addr, r->addr->ai_addrlen); + if (rc == 0) { + request_tls_start(r); + return; + } + if (errno == EINPROGRESS) { + r->phase = REQ_CONNECT; + request_schedule_io(r, EV_WRITE, API_TIMEOUT); + return; + } + saved = errno; + close(fd); + r->http.fd = -1; + r->addr = r->addr->ai_next; + } + (void)snprintf(r->lasterr, sizeof(r->lasterr), "connect %s: %s", + v->host, strerror(saved != 0 ? saved : ECONNREFUSED)); + request_retry(r, r->lasterr, -1); +} + +static void +request_dns_done(struct asr_result *ar, void *arg) +{ + struct api_request *r = arg; + struct prov *v = &r->api->provs[r->slot]; + + r->resolver = NULL; + request_timer_clear(r); + if (ar->ar_gai_errno != 0 || ar->ar_addrinfo == NULL) { + (void)snprintf(r->lasterr, sizeof(r->lasterr), "%s: %s", + v->host, gai_strerror(ar->ar_gai_errno)); + request_retry(r, r->lasterr, -1); + return; + } + r->addrs = ar->ar_addrinfo; + r->addr = r->addrs; + request_connect_next(r); +} + +static void +request_start_attempt(struct api_request *r) +{ + struct addrinfo hints; + struct asr_query *aq; + struct prov *v = &r->api->provs[r->slot]; + + request_attempt_clear(r); + secret_streams_clear(r); + r->attempt++; + r->terminal = 0; + r->retryable = 0; + r->retry_after = -1; + r->response_text = 0; + r->response_bytes = 0; + r->tool_args_total = 0; + memset(&r->usage, 0, sizeof(r->usage)); + http_init(&r->http); + r->http_live = 1; + r->http.host = xstrdup(v->host); + r->http.port = xstrdup(v->port); + memset(&hints, 0, sizeof(hints)); + hints.ai_family = AF_UNSPEC; + hints.ai_socktype = SOCK_STREAM; + aq = getaddrinfo_async(v->host, v->port, &hints, NULL); + if (aq == NULL) { + request_retry(r, "cannot start provider resolution", -1); + return; + } + r->resolver = event_asr_run(aq, request_dns_done, r); + if (r->resolver == NULL) { + asr_abort(aq); + request_retry(r, "cannot monitor provider resolution", -1); + return; + } + r->phase = REQ_DNS; + request_deadline_in(r, API_TIMEOUT); + request_schedule_timer(r, API_TIMEOUT); +} + +static void +request_timer(int fd, short event, void *arg) +{ + struct api_request *r = arg; + + (void)fd; + (void)event; + r->timer_added = 0; + if (r->paused) { + r->paused = 0; + if (request_deadline_left(r) == 0 || + (r->phase == REQ_BODY && + request_total_deadline_left(r) == 0)) { + if (r->phase == REQ_ERROR_BODY) + request_error_done(r); + else + request_fail(r, "provider response timed out"); + return; + } + if (imsgbuf_queuelen(&r->worker->iev.ibuf) > + API_IPC_LOW_WATER) { + request_pause(r); + return; + } + request_read(r); + return; + } + if (r->phase == REQ_BACKOFF) { + if (r->api->ninflight >= API_NETWORK_MAX) { + r->phase = REQ_QUEUED; + return; + } + r->api->ninflight++; + r->network_slot = 1; + request_start_attempt(r); + return; + } + if (r->phase == REQ_DNS) { + request_retry(r, "provider resolution timed out", -1); + return; + } + fatalx("fugu-api: stray request timer"); +} + +static void +request_io(int fd, short event, void *arg) +{ + struct api_request *r = arg; + int error; + socklen_t errorlen; + + (void)fd; + r->io_added = 0; + if (event & EV_TIMEOUT) { + if (r->phase == REQ_CONNECT || r->phase == REQ_HANDSHAKE) + request_retry(r, "provider connection timed out", -1); + else if (r->phase == REQ_ERROR_BODY) + request_error_done(r); + else + request_fail(r, "provider response timed out"); + return; + } + switch (r->phase) { + case REQ_CONNECT: + error = 0; + errorlen = sizeof(error); + if (getsockopt(r->http.fd, SOL_SOCKET, SO_ERROR, &error, + &errorlen) == -1) + error = errno; + if (error != 0) { + close(r->http.fd); + r->http.fd = -1; + r->addr = r->addr->ai_next; + request_connect_next(r); + } else + request_tls_start(r); + break; + case REQ_HANDSHAKE: + request_handshake(r); + break; + case REQ_WRITE: + request_write(r); + break; + case REQ_HEADERS: + case REQ_BODY: + case REQ_ERROR_BODY: + request_read(r); + break; + default: + fatalx("fugu-api: request I/O in phase %d", r->phase); + } +} + +static struct api_request * +request_new(struct worker *w, uint32_t id) +{ + struct api *a = w->ctx; + struct api_request *r; + + if (id == 0 || a->nrequests >= API_REQUEST_MAX || + request_find(a, id) != NULL) + fatalx("fugu-api: invalid or duplicate request id"); + r = xcalloc(1, sizeof(*r)); + r->api = a; + r->worker = w; + r->id = id; + r->phase = REQ_ASSEMBLY; + buf_init(&r->body); + buf_init(&r->wire); + buf_init(&r->errbody); + TAILQ_INSERT_TAIL(&a->requests, r, entry); + a->nrequests++; + return (r); +} + /* * One provider's model listing (behavior.md 2.1, 8): GET the per-type * models endpoint and stream each id back tagged with the slot. The @@ -1045,19 +1712,24 @@ list_one(struct worker *w, struct api *a, int slot) char headers[API_AUTH_HDR_MAX]; char tmp[8192]; ssize_t n; - int data, i, root, nmodels, sent = 0; + int data, i, root, nmodels, rc, sent = 0; if (v->key == NULL) return; - if (auth_headers(v, headers, sizeof(headers)) == -1) + if (request_auth_headers(v, headers, sizeof(headers)) == -1) { + explicit_bzero(headers, sizeof(headers)); return; + } if (http_connect(&h, v->host, v->port, v->host, a->tlscfg, LIST_TIMEOUT, NULL, NULL) == -1) { + explicit_bzero(headers, sizeof(headers)); http_close(&h); return; } - if (http_request(&h, "GET", "/v1/models", headers, NULL, 0, - LIST_TIMEOUT) == -1 || http_response(&h, LIST_TIMEOUT) == -1 || + rc = http_request(&h, "GET", "/v1/models", headers, NULL, 0, + LIST_TIMEOUT); + explicit_bzero(headers, sizeof(headers)); + if (rc == -1 || http_response(&h, LIST_TIMEOUT) == -1 || h.status < 200 || h.status > 299) { http_close(&h); return; @@ -1091,7 +1763,7 @@ list_one(struct worker *w, struct api *a, int slot) http_close(&h); if (json_parse(&j, body.data, body.len, 0) != 0) { - buf_free(&body); + buf_freezero(&body); return; } root = json_root(&j); @@ -1102,9 +1774,10 @@ list_one(struct worker *w, struct api *a, int slot) struct model_ent me; char *id; size_t idl; + int obj; - id = json_get_str(&j, json_obj_get(&j, - json_arr_get(&j, data, i), "id"), &idl); + obj = json_arr_get(&j, data, i); + id = json_get_str(&j, json_obj_get(&j, obj, "id"), &idl); if (id == NULL) continue; if (idl == 0 || idl > MODEL_ID_MAX || strlen(id) != idl || @@ -1114,50 +1787,126 @@ list_one(struct worker *w, struct api *a, int slot) } memset(&me, 0, sizeof(me)); me.slot = slot; + me.context_window = model_window_parse(&j, obj); buf_init(&e); buf_add(&e, &me, sizeof(me)); buf_addstr(&e, v->name); buf_addc(&e, '\0'); buf_add(&e, id, idl); - relay(w, FUGU_IMSG_MODEL, e.data, e.len); + relay_id(w, 0, FUGU_IMSG_MODEL, e.data, e.len); buf_free(&e); free(id); sent++; } json_done(&j); - buf_free(&body); + buf_freezero(&body); } +static void +list_all(struct worker *w, struct api *a) +{ + size_t slot; + + for (slot = 0; slot < a->nprov; slot++) + list_one(w, a, (int)slot); + relay_id(w, 0, FUGU_IMSG_MODELS_END, NULL, 0); +} + static int api_dispatch(struct worker *w, struct imsg *imsg) { struct api *a = w->ctx; + struct api_request *r; size_t len; switch (imsg_get_type(imsg)) { case FUGU_IMSG_REQUEST: { const u_char *p = imsg_bytes(imsg, &len); + uint32_t id = imsg_get_id(imsg); + size_t cap, growth; + if ((r = request_find(a, id)) == NULL) + r = request_new(w, id); + if (r->phase != REQ_ASSEMBLY && r->phase != REQ_REJECTED) + fatalx("fugu-api: request chunk after request end"); + /* the coordinator checks before sending; an overrun * here is a protocol violation, not a user error */ - if (a->body.len + len > FUGU_REQUEST_MAX) + if (len > FUGU_REQUEST_MAX - r->received) fatalx("fugu-api: request too large"); - if (p != NULL && len > 0) - buf_add(&a->body, p, len); + r->received += len; + if (r->phase == REQ_REJECTED) + return (0); + cap = buf_capacity_after(&r->body, len); + growth = cap - r->body.cap; + if (a->request_memory > API_REQUEST_MEMORY_MAX || + growth > API_REQUEST_MEMORY_MAX - a->request_memory) { + if (r->body.cap > a->request_memory) + fatalx("fugu-api: request memory accounting underflow"); + a->request_memory -= r->body.cap; + buf_free(&r->body); + buf_init(&r->body); + r->phase = REQ_REJECTED; + return (0); + } + if (p != NULL && len > 0) { + buf_add(&r->body, p, len); + a->request_memory += growth; + } return (0); } - case FUGU_IMSG_REQUEST_END: - api_run(w); + case FUGU_IMSG_REQUEST_END: { + struct api_request_end end; + uint32_t id = imsg_get_id(imsg); + + if ((r = request_find(a, id)) == NULL) + fatalx("fugu-api: REQUEST_END without matching request"); + if (r->phase == REQ_REJECTED) { + relay_error(r, + "aggregate provider request memory bound exceeded"); + request_free(r); + return (0); + } + if (r->phase != REQ_ASSEMBLY || r->body.len == 0) + fatalx("fugu-api: REQUEST_END without matching request"); + if (imsg_get_len(imsg) == 0) + r->slot = a->active; + else { + if (imsg_get_len(imsg) != sizeof(end) || + imsg_get_data(imsg, &end, sizeof(end)) == -1 || + end.provider_slot < 0 || + (size_t)end.provider_slot >= a->nprov) + fatalx("fugu-api: bad request provider slot"); + r->slot = (size_t)end.provider_slot; + } + if (a->provs[r->slot].key == NULL) { + relay_error(r, "no API key configured"); + request_free(r); + } else if (a->ninflight < API_NETWORK_MAX) { + a->ninflight++; + r->network_slot = 1; + request_start_attempt(r); + } else + r->phase = REQ_QUEUED; return (0); + } + case FUGU_IMSG_REQUEST_CANCEL: + if (imsg_get_len(imsg) != 0) + fatalx("fugu-api: bad REQUEST_CANCEL"); + if ((r = request_find(a, imsg_get_id(imsg))) != NULL) { + relay_error(r, "request cancelled"); + request_free(r); + } + return (0); case FUGU_IMSG_PROV: prov_add(a, imsg); return (0); case FUGU_IMSG_LIST_MODELS: { - size_t slot; - - for (slot = 0; slot < a->nprov; slot++) - list_one(w, a, (int)slot); - relay(w, FUGU_IMSG_MODELS_END, NULL, 0); + if (a->nrequests != 0) { + a->list_pending = 1; + return (0); + } + list_all(w, a); return (0); } case FUGU_IMSG_SET_PROVIDER: { @@ -1186,8 +1935,8 @@ main(int argc, char *argv[]) if (tls_init() == -1) fatalx("tls_init"); - buf_init(&a.body); a.retry_ms = 1000; + TAILQ_INIT(&a.requests); /* regress compresses the pinned backoff; ignored when setgid */ if (!issetugid()) { const char *e, *errstr; blob - d0562781018e69ca18e9f0e1a00fbf8a1e80cb3c blob + b8171df9edc809073f9da40dde92cfca76440d02 --- src/fugu-tool/Makefile +++ src/fugu-tool/Makefile @@ -1,6 +1,6 @@ PROG= fugu-tool -SRCS= main.c tools.c job.c supervisor.c worker.c imsgev.c \ - json.c buf.c diff.c log.c xmalloc.c +SRCS= main.c agent.c tools.c job.c supervisor.c worker.c imsgev.c \ + anthropic_req.c openai_req.c msg.c json.c buf.c diff.c log.c xmalloc.c BINDIR= ${LIBEXECDIR}/fugu MAN= blob - /dev/null blob + d37bcaf8fb50b8ec2b9e2ed1e60476b21a7b9895 (mode 644) --- /dev/null +++ src/fugu-tool/agent.c @@ -0,0 +1,889 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * The ephemeral subagent loop lives inside fugu-tool -r. That gives every + * local command the I15 kernel ceiling (tree rx, no write/create, no network) + * while this process owns no model or web credential and no Journal. It + * builds a fresh canonical conversation, asks the Coordinator to broker + * provider/web traffic, and returns only its final report. + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "log.h" +#include "xmalloc.h" +#include "buf.h" +#include "json.h" +#include "msg.h" +#include "anthropic.h" +#include "openai.h" +#include "worker.h" +#include "proto.h" +#include "tools.h" +#include "agent.h" + +#define AGENT_ITER_MAX 25 +#define AGENT_CALL_MAX 64 +#define AGENT_PROGRESS_MAX 256 + +static const char agent_limited_empty[] = + "Generation stopped before emitting content."; + +#define OBJ(props, req) "{\"type\":\"object\",\"properties\":" props \ + ",\"required\":" req "}" + +struct agent_tcall { + int index; + char id[256]; + char name[FUGU_TOOL_NAME_MAX]; + struct buf input; + size_t input_total; + int malformed; +}; + +struct agent_state { + struct worker *w; + struct tool_ctx *tools; + struct agent_boot *boot; + struct msglist conv; + int prev_msgs; + struct buf text; + struct buf report; + struct agent_tcall calls[AGENT_CALL_MAX]; + int ncalls; + char stop[FUGU_STOP_REASON_MAX]; + char err[FUGU_ERROR_MAX]; + size_t arg_total; + size_t data_total; + int data_exceeded; + int iterations; + int truncated; + size_t report_tail; + int have_report_tail; +}; + +static const struct tool_def agent_read = { + "read", + "Return file contents. Use offset and limit for bounded line windows; " + "request only the part needed.", + OBJ("{\"path\":{\"type\":\"string\"}," + "\"offset\":{\"type\":\"integer\"}," + "\"limit\":{\"type\":\"integer\"}}", "[\"path\"]") +}; + +static const struct tool_def agent_shell = { + "shell", + "Run /bin/ksh -c read-only in the project. The command has no network " + "and cannot write or create files; use it only for inspection and tests " + "that do not mutate the tree.", + OBJ("{\"command\":{\"type\":\"string\"}}", "[\"command\"]") +}; + +static const struct tool_def agent_grep = { + "grep", "Search file contents recursively with line numbers.", + OBJ("{\"pattern\":{\"type\":\"string\"}," + "\"path\":{\"type\":\"string\"}}", "[\"pattern\"]") +}; + +static const struct tool_def agent_find = { + "find", "List paths below a directory, optionally filtered by name.", + OBJ("{\"path\":{\"type\":\"string\"}," + "\"name\":{\"type\":\"string\"}}", "[]") +}; + +static const struct tool_def agent_ls = { + "ls", "List a directory in long form.", + OBJ("{\"path\":{\"type\":\"string\"}}", "[]") +}; + +static const struct tool_def agent_web_search = { + "web_search", "Search the web through the configured credential broker.", + OBJ("{\"query\":{\"type\":\"string\"}}", "[\"query\"]") +}; + +static const struct tool_def agent_web_fetch = { + "web_fetch", "Fetch one public HTTPS URL through the confined web broker.", + OBJ("{\"url\":{\"type\":\"string\"}}", "[\"url\"]") +}; + +static const struct tool_def agent_http_request = { + "http_request", "Send an HTTPS request to an operator-allowed endpoint.", + OBJ("{\"url\":{\"type\":\"string\"}," + "\"method\":{\"type\":\"string\"}," + "\"headers\":{\"type\":\"object\"," + "\"additionalProperties\":{\"type\":\"string\"}}," + "\"body\":{\"type\":\"string\"}}", "[\"url\"]") +}; + +static const struct tool_def * +agent_tooldefs(const struct agent_state *a, int *np) +{ + static struct tool_def defs[8]; + int i = 0; + + defs[i++] = agent_read; + defs[i++] = agent_shell; + defs[i++] = agent_grep; + defs[i++] = agent_find; + defs[i++] = agent_ls; + if (a->boot->start.web_search) { + defs[i++] = agent_web_search; + defs[i++] = agent_web_fetch; + } + if (a->boot->start.http_request) + defs[i++] = agent_http_request; + *np = i; + return (defs); +} + +void +agent_boot_init(struct agent_boot *b) +{ + memset(b, 0, sizeof(*b)); + buf_init(&b->system); + buf_init(&b->prompt); +} + +void +agent_boot_free(struct agent_boot *b) +{ + buf_free(&b->system); + buf_free(&b->prompt); + memset(b, 0, sizeof(*b)); +} + +int +agent_boot_start(struct agent_boot *b, const void *data, size_t len) +{ + const u_char *p = data; + const char *model, *label; + size_t left, ml, ll; + + if (b->have_start || p == NULL || len <= sizeof(b->start) + 1) + return (-1); + memcpy(&b->start, p, sizeof(b->start)); + p += sizeof(b->start); + left = len - sizeof(b->start); + model = (const char *)p; + ml = strnlen(model, left); + if (ml == 0 || ml > FUGU_MODEL_ID_MAX || ml == left) + return (-1); + label = model + ml + 1; + left -= ml + 1; + ll = strnlen(label, left); + if (ll > FUGU_AGENT_LABEL_MAX || ll == left || ll + 1 != left || + (b->start.provider_type != PROVIDER_ANTHROPIC && + b->start.provider_type != PROVIDER_OPENAI) || + b->start.provider_slot < 0 || b->start.max_tokens < 1 || + (b->start.web_search != 0 && b->start.web_search != 1) || + (b->start.http_request != 0 && b->start.http_request != 1)) + return (-1); + memcpy(b->model, model, ml + 1); + memcpy(b->label, label, ll + 1); + b->have_start = 1; + return (0); +} + +int +agent_boot_system(struct agent_boot *b, const void *data, size_t len) +{ + if (!b->have_start || len > FUGU_TURN_DATA_MAX - b->system.len) + return (-1); + if (data != NULL && len > 0) + buf_add(&b->system, data, len); + return (0); +} + +int +agent_boot_prompt(struct agent_boot *b, const void *data, size_t len) +{ + if (!b->have_start || len > FUGU_TURN_DATA_MAX - b->prompt.len) + return (-1); + if (data != NULL && len > 0) + buf_add(&b->prompt, data, len); + return (0); +} + +static const u_char * +frame_bytes(struct imsg *imsg, size_t *lenp) +{ + struct ibuf ib; + + if (imsg_get_len(imsg) == 0 || imsg_get_ibuf(imsg, &ib) == -1) { + *lenp = 0; + return (NULL); + } + *lenp = ibuf_size(&ib); + return (ibuf_data(&ib)); +} + +static void +agent_send(struct agent_state *a, uint32_t type, const void *data, size_t len) +{ + struct pollfd pfd; + + if (imsg_compose(&a->w->iev.ibuf, type, 0, -1, -1, data, len) == -1) + fatal("agent imsg_compose"); + while (imsgbuf_queuelen(&a->w->iev.ibuf) > 0) { + if (imsgbuf_write(&a->w->iev.ibuf) == 0) + continue; + if (errno != EAGAIN && errno != EINTR) + fatal("agent imsgbuf_write"); + memset(&pfd, 0, sizeof(pfd)); + pfd.fd = a->w->iev.ibuf.fd; + pfd.events = POLLOUT; + if (poll(&pfd, 1, -1) == -1 && errno != EINTR) + fatal("agent output poll"); + } +} + +static void +agent_send_bytes(struct agent_state *a, uint32_t type, const void *data, + size_t len) +{ + const u_char *p = data; + size_t off, n; + + for (off = 0; off < len; off += n) { + n = len - off; + if (n > FUGU_CHUNK) + n = FUGU_CHUNK; + agent_send(a, type, p + off, n); + } +} + +static void agent_progress(struct agent_state *, const char *, ...) + __attribute__((__format__ (printf, 2, 3))); + +static void +agent_progress(struct agent_state *a, const char *fmt, ...) +{ + char buf[AGENT_PROGRESS_MAX]; + va_list ap; + int i, n; + + va_start(ap, fmt); + n = vsnprintf(buf, sizeof(buf), fmt, ap); + va_end(ap); + if (n < 0) + return; + if ((size_t)n >= sizeof(buf)) + n = sizeof(buf) - 1; + /* Progress is terminal-visible. Keep normal hostile provider/config + * strings from turning a status frame into terminal control; the + * coordinator independently validates this boundary against a + * compromised child. */ + for (i = 0; i < n; i++) + if ((u_char)buf[i] < 0x20 || (u_char)buf[i] > 0x7e) + buf[i] = '?'; + agent_send(a, FUGU_IMSG_AGENT_PROGRESS, buf, (size_t)n); +} + +/* Blocking only inside the ephemeral -r worker. Provider/web deadlines live + * in their credential custodians; SIGTERM cancels this process and any shell + * group through fugu-tool's existing signal hook. */ +static int +agent_next(struct agent_state *a, struct imsg *imsg) +{ + struct pollfd pfd; + ssize_t n; + + for (;;) { + if ((n = imsg_get(&a->w->iev.ibuf, imsg)) == -1) + fatal("agent imsg_get"); + if (n > 0) + return (1); + pfd.fd = a->w->iev.ibuf.fd; + pfd.events = POLLIN; + do { + n = poll(&pfd, 1, -1); + } while (n == -1 && (errno == EINTR || errno == EAGAIN)); + if (n == -1) + fatal("agent poll"); + if ((n = imsgbuf_read(&a->w->iev.ibuf)) == -1) + fatal("agent imsgbuf_read"); + if (n == 0) + return (0); + } +} + +static int +agent_msg_count(const struct msglist *l) +{ + struct msg *m; + int n = 0; + + TAILQ_FOREACH(m, l, entry) + n++; + return (n); +} + +static void +agent_build_request(struct agent_state *a, struct buf *body, + const struct tool_def *defs, int ndefs) +{ + if (a->boot->start.provider_type == PROVIDER_OPENAI) { + struct openai_req req; + + memset(&req, 0, sizeof(req)); + req.model = a->boot->model; + req.max_tokens = a->boot->start.max_tokens; + req.system = (const char *)a->boot->system.data; + req.systemlen = a->boot->system.len; + req.tools = defs; + req.ntools = ndefs; + req.conv = &a->conv; + openai_build_request(body, &req); + } else { + struct anthropic_req req; + + memset(&req, 0, sizeof(req)); + req.model = a->boot->model; + req.max_tokens = a->boot->start.max_tokens; + req.system = (const char *)a->boot->system.data; + req.systemlen = a->boot->system.len; + req.tools = defs; + req.ntools = ndefs; + req.conv = &a->conv; + req.cache = 1; + req.cache_tail_msg = a->prev_msgs > 0 ? a->prev_msgs - 1 : -1; + anthropic_build_request(body, &req); + } +} + +static struct agent_tcall * +agent_call_find(struct agent_state *a, int index) +{ + int i; + + for (i = 0; i < a->ncalls; i++) + if (a->calls[i].index == index) + return (&a->calls[i]); + return (NULL); +} + +static void +agent_calls_reset(struct agent_state *a) +{ + int i; + + for (i = 0; i < a->ncalls; i++) + buf_free(&a->calls[i].input); + a->ncalls = 0; + a->arg_total = 0; +} + +static int +agent_account(struct agent_state *a, size_t len) +{ + if (len > FUGU_TURN_DATA_MAX - a->data_total) { + a->data_exceeded = 1; + return (-1); + } + a->data_total += len; + return (0); +} + +/* Send one projection and collect its already-decoded provider events. */ +static int +agent_generation(struct agent_state *a, const struct tool_def *defs, + int ndefs) +{ + struct buf body; + struct imsg imsg; + const u_char *p; + size_t len; + uint32_t type; + + agent_calls_reset(a); + buf_reset(&a->text); + a->stop[0] = '\0'; + a->err[0] = '\0'; + buf_init(&body); + agent_build_request(a, &body, defs, ndefs); + if (body.len > FUGU_REQUEST_MAX) { + buf_free(&body); + strlcpy(a->err, "subagent request exceeds 16 MiB", sizeof(a->err)); + return (-1); + } + agent_progress(a, "requesting %s", a->boot->model); + agent_send_bytes(a, FUGU_IMSG_AGENT_REQUEST, body.data, body.len); + agent_send(a, FUGU_IMSG_AGENT_REQUEST_END, NULL, 0); + buf_free(&body); + a->prev_msgs = agent_msg_count(&a->conv); + + while (agent_next(a, &imsg)) { + type = imsg_get_type(&imsg); + p = frame_bytes(&imsg, &len); + switch (type) { + case FUGU_IMSG_A_TEXT: + if (len > FUGU_RESPONSE_MAX - a->text.len) { + strlcpy(a->err, "subagent response exceeds 16 MiB", + sizeof(a->err)); + imsg_free(&imsg); + return (-1); + } + if (p != NULL && len > 0 && + agent_account(a, len) == 0) + buf_add(&a->text, p, len); + break; + case FUGU_IMSG_A_TOOL_BEGIN: { + struct a_tool_begin h; + struct agent_tcall *tc; + const char *id, *name; + size_t il, nl; + + if (p == NULL || len < sizeof(h) || + a->ncalls >= AGENT_CALL_MAX) + fatalx("subagent malformed tool begin"); + memcpy(&h, p, sizeof(h)); + id = (const char *)p + sizeof(h); + il = strnlen(id, len - sizeof(h)); + if (il == 0 || il == len - sizeof(h) || il >= 256) + fatalx("subagent malformed tool id"); + name = id + il + 1; + nl = len - sizeof(h) - il - 1; + if (h.index < 0 || agent_call_find(a, h.index) != NULL || + nl == 0 || nl >= FUGU_TOOL_NAME_MAX || + memchr(name, '\0', nl) != NULL) + fatalx("subagent malformed tool name"); + tc = &a->calls[a->ncalls++]; + memset(tc, 0, sizeof(*tc)); + tc->index = h.index; + memcpy(tc->id, id, il + 1); + memcpy(tc->name, name, nl); + tc->name[nl] = '\0'; + buf_init(&tc->input); + break; + } + case FUGU_IMSG_A_TOOL_INPUT: { + struct agent_tcall *tc; + int idx; + + if (p == NULL || len < sizeof(idx)) + fatalx("subagent malformed tool input"); + memcpy(&idx, p, sizeof(idx)); + tc = agent_call_find(a, idx); + if (tc == NULL || len - sizeof(idx) > + FUGU_TOOL_ARG_MAX - tc->input_total || len - sizeof(idx) > + FUGU_TOOL_ARGS_TOTAL_MAX - a->arg_total) + fatalx("subagent excessive tool input"); + len -= sizeof(idx); + if (agent_account(a, len) == 0) + buf_add(&tc->input, p + sizeof(idx), len); + tc->input_total += len; + a->arg_total += len; + break; + } + case FUGU_IMSG_A_USAGE: + if (p == NULL || len != sizeof(struct a_usage)) + fatalx("subagent malformed usage"); + agent_send(a, FUGU_IMSG_AGENT_USAGE, p, len); + break; + case FUGU_IMSG_A_RETRY: + agent_progress(a, "provider retry"); + break; + case FUGU_IMSG_A_DONE: + if (p == NULL || len == 0 || len >= sizeof(a->stop) || + memchr(p, '\0', len) != NULL) + fatalx("subagent malformed stop reason"); + memcpy(a->stop, p, len); + a->stop[len] = '\0'; + imsg_free(&imsg); + if (a->data_exceeded) { + strlcpy(a->err, "subagent data bound exceeded", + sizeof(a->err)); + return (-1); + } + return (0); + case FUGU_IMSG_A_ERROR: + if (p == NULL || len == 0 || len >= sizeof(a->err) || + memchr(p, '\0', len) != NULL) + fatalx("subagent malformed provider error"); + memcpy(a->err, p, len); + a->err[len] = '\0'; + imsg_free(&imsg); + return (-1); + case FUGU_IMSG_SHUTDOWN: + imsg_free(&imsg); + exit(0); + default: + fatalx("subagent unexpected imsg %u", type); + } + imsg_free(&imsg); + } + strlcpy(a->err, "coordinator closed the subagent channel", sizeof(a->err)); + return (-1); +} + +static int +agent_is_web(const char *name) +{ + return (strcmp(name, "web_search") == 0 || + strcmp(name, "web_fetch") == 0 || + strcmp(name, "http_request") == 0); +} + +static int +agent_local_allowed(const char *name) +{ + return (strcmp(name, "read") == 0 || strcmp(name, "shell") == 0 || + strcmp(name, "grep") == 0 || strcmp(name, "find") == 0 || + strcmp(name, "ls") == 0); +} + +static void +agent_web(struct agent_state *a, struct agent_tcall *tc, struct buf *out, + int *is_error) +{ + struct tool_req req; + struct tool_result result; + struct imsg imsg; + const u_char *p; + size_t len; + + memset(&req, 0, sizeof(req)); + strlcpy(req.name, tc->name, sizeof(req.name)); + agent_send(a, FUGU_IMSG_AGENT_WEB_REQ, &req, sizeof(req)); + agent_send_bytes(a, FUGU_IMSG_AGENT_WEB_ARG, tc->input.data, + tc->input.len); + agent_send(a, FUGU_IMSG_AGENT_WEB_RUN, NULL, 0); + buf_reset(out); + *is_error = 1; + while (agent_next(a, &imsg)) { + p = frame_bytes(&imsg, &len); + switch (imsg_get_type(&imsg)) { + case FUGU_IMSG_AGENT_WEB_OUT: + if (p == NULL || len == 0 || len > FUGU_WEB_OUT_MAX - out->len) + fatalx("subagent web result overrun"); + buf_add(out, p, len); + break; + case FUGU_IMSG_AGENT_WEB_RESULT: + if (p == NULL || len != sizeof(result)) + fatalx("subagent malformed web result"); + memcpy(&result, p, sizeof(result)); + if (result.is_error != 0 && result.is_error != 1) + fatalx("subagent invalid web result"); + *is_error = result.is_error; + imsg_free(&imsg); + return; + case FUGU_IMSG_SHUTDOWN: + imsg_free(&imsg); + exit(0); + default: + fatalx("subagent unexpected web imsg %u", + imsg_get_type(&imsg)); + } + imsg_free(&imsg); + } + exit(0); +} + +static struct msg * +agent_assistant_message(struct agent_state *a) +{ + struct msg *m; + int i; + + m = msg_new(ROLE_ASSISTANT); + if (a->text.len > 0) + msg_add_text(m, (const char *)a->text.data, a->text.len); + else if (a->ncalls == 0 && + (strcmp(a->stop, "max_tokens") == 0 || + strcmp(a->stop, "length") == 0)) + msg_add_text(m, agent_limited_empty, + sizeof(agent_limited_empty) - 1); + for (i = 0; i < a->ncalls; i++) { + if (a->calls[i].input.len == 0) + buf_addstr(&a->calls[i].input, "{}"); + else if (!json_valid_object(a->calls[i].input.data, + a->calls[i].input.len)) { + buf_reset(&a->calls[i].input); + buf_addstr(&a->calls[i].input, "{}"); + a->calls[i].malformed = 1; + } + msg_add_tool_use(m, a->calls[i].id, a->calls[i].name, + (const char *)a->calls[i].input.data, a->calls[i].input.len); + } + return (m); +} + +static void +agent_account_result(struct agent_state *a, struct buf *out, int *is_error) +{ + static const char mark[] = "subagent data bound exceeded"; + size_t room, n; + + if (out->len > FUGU_TOOL_OUT_MAX) { + out->len = FUGU_TOOL_OUT_MAX; + *is_error = 1; + } + if (agent_account(a, out->len) == 0) + return; + room = FUGU_TURN_DATA_MAX - a->data_total; + buf_reset(out); + n = sizeof(mark) - 1; + if (n > room) + n = room; + if (n > 0) + buf_add(out, mark, n); + a->data_total += n; + *is_error = 1; +} + +static struct msg * +agent_execute_calls(struct agent_state *a) +{ + struct msg *m; + struct buf out; + int i, is_error; + + m = msg_new(ROLE_USER); + buf_init(&out); + for (i = 0; i < a->ncalls; i++) { + struct agent_tcall *tc = &a->calls[i]; + + if (a->data_exceeded) { + buf_reset(&out); + buf_addstr(&out, "subagent data bound exceeded"); + is_error = 1; + } + if (!a->data_exceeded && tc->malformed) { + buf_reset(&out); + buf_addstr(&out, "subagent: malformed tool arguments"); + is_error = 1; + } else if (!a->data_exceeded && agent_local_allowed(tc->name)) { + agent_progress(a, "tool %s", tc->name); + tool_exec(a->tools, tc->name, tc->input.data, tc->input.len, + &out, &is_error); + } else if (!a->data_exceeded && agent_is_web(tc->name) && + ((strcmp(tc->name, "http_request") == 0 && + a->boot->start.http_request) || + (strcmp(tc->name, "http_request") != 0 && + a->boot->start.web_search))) { + agent_progress(a, "tool %s", tc->name); + agent_web(a, tc, &out, &is_error); + } else if (!a->data_exceeded) { + buf_reset(&out); + buf_addf(&out, "%s is unavailable to a read-only subagent", + tc->name); + is_error = 1; + } + agent_account_result(a, &out, &is_error); + msg_add_tool_result(m, tc->id, (const char *)out.data, out.len, + is_error); + } + buf_free(&out); + return (m); +} + +static struct msg * +agent_answer_dangling(struct agent_state *a) +{ + static const char reason[] = + "subagent tool call was truncated and was not executed"; + struct msg *m; + struct buf out; + int i, is_error; + + m = msg_new(ROLE_USER); + buf_init(&out); + for (i = 0; i < a->ncalls; i++) { + buf_reset(&out); + buf_add(&out, reason, sizeof(reason) - 1); + is_error = 1; + agent_account_result(a, &out, &is_error); + msg_add_tool_result(m, a->calls[i].id, + (const char *)out.data, out.len, is_error); + } + buf_free(&out); + return (m); +} + +static int +agent_stop_limited(const struct agent_state *a) +{ + return (strcmp(a->stop, "max_tokens") == 0 || + strcmp(a->stop, "length") == 0); +} + +static void +agent_report_text(struct agent_state *a) +{ + if (a->text.len == 0) + return; + if (a->report.len > 0) + buf_addstr(&a->report, "\n\n"); + buf_add(&a->report, a->text.data, a->text.len); +} + +static void +agent_report_tail(struct agent_state *a) +{ + a->report_tail = a->report.len; + a->have_report_tail = 1; +} + +static void +agent_mark_truncated(struct agent_state *a) +{ + static const char mark[] = + "\n\n[fugu: subagent report truncated by its execution bound]"; + struct buf bounded; + size_t marklen = sizeof(mark) - 1; + size_t payload = FUGU_TOOL_OUT_MAX - marklen; + size_t tail, keep_tail, keep_head; + + a->truncated = 1; + if (a->report.len > payload && a->have_report_tail && + a->report_tail < a->report.len) { + tail = a->report.len - a->report_tail; + keep_tail = tail < payload ? tail : payload; + keep_head = payload - keep_tail; + if (keep_head > a->report_tail) + keep_head = a->report_tail; + buf_init(&bounded); + if (keep_head > 0) + buf_add(&bounded, a->report.data, keep_head); + if (keep_tail > 0) + buf_add(&bounded, a->report.data + a->report.len - keep_tail, + keep_tail); + buf_free(&a->report); + a->report = bounded; + } else if (a->report.len > payload) + a->report.len = payload; + buf_add(&a->report, mark, marklen); +} + +static __dead void +agent_finish(struct agent_state *a, int is_error) +{ + struct agent_result r; + + if (a->report.len == 0) + buf_addstr(&a->report, is_error ? "subagent failed without a report" : + "subagent completed without text"); + if (a->report.len > FUGU_TOOL_OUT_MAX) + agent_mark_truncated(a); + agent_send_bytes(a, FUGU_IMSG_AGENT_OUT, a->report.data, a->report.len); + memset(&r, 0, sizeof(r)); + r.is_error = is_error; + r.truncated = a->truncated; + r.iterations = a->iterations; + agent_send(a, FUGU_IMSG_AGENT_RESULT, &r, sizeof(r)); + exit(0); +} + +static __dead void +agent_conclude(struct agent_state *a, struct msg *m) +{ + static const char conclude[] = + "You have reached the subagent tool-round bound or another subagent " + "execution bound. Do not call tools. " + "Conclude now with the most useful concise final report possible, " + "including uncertainty and unfinished checks."; + + msg_add_text(m, conclude, sizeof(conclude) - 1); + agent_report_tail(a); + if (agent_generation(a, NULL, 0) == 0) + agent_report_text(a); + else if (a->err[0] != '\0') { + if (a->report.len > 0) + buf_addstr(&a->report, "\n\n"); + buf_addf(&a->report, "subagent conclusion failed: %s", a->err); + } + agent_mark_truncated(a); + agent_finish(a, 0); +} + +__dead void +agent_run(struct worker *w, struct tool_ctx *tools, struct agent_boot *boot) +{ + struct agent_state a; + const struct tool_def *defs; + struct msg *m; + int ndefs, round; + + memset(&a, 0, sizeof(a)); + a.w = w; + a.tools = tools; + a.boot = boot; + TAILQ_INIT(&a.conv); + buf_init(&a.text); + buf_init(&a.report); + a.data_total = boot->prompt.len; + m = msg_new(ROLE_USER); + msg_add_text(m, (const char *)boot->prompt.data, boot->prompt.len); + TAILQ_INSERT_TAIL(&a.conv, m, entry); + defs = agent_tooldefs(&a, &ndefs); + agent_progress(&a, "started"); + + for (round = 1; round <= AGENT_ITER_MAX; round++) { + if (agent_generation(&a, defs, ndefs) == -1) { + agent_report_tail(&a); + if (a.report.len > 0) + buf_addstr(&a.report, "\n\n"); + buf_addf(&a.report, "subagent provider error: %s", a.err); + agent_finish(&a, 1); + } + m = agent_assistant_message(&a); + TAILQ_INSERT_TAIL(&a.conv, m, entry); + if (agent_stop_limited(&a)) { + agent_report_text(&a); + m = agent_answer_dangling(&a); + TAILQ_INSERT_TAIL(&a.conv, m, entry); + agent_conclude(&a, m); + } + if (a.ncalls == 0) { + /* A normal completion returns only its final assistant text; + * interim prose is retained solely as a cap fallback. */ + buf_reset(&a.report); + agent_report_text(&a); + agent_finish(&a, 0); + } + /* Preserve tool-bearing prose so an iteration cap can still return + * everything useful emitted before its one conclusion attempt. */ + agent_report_text(&a); + + a.iterations++; + m = agent_execute_calls(&a); + TAILQ_INSERT_TAIL(&a.conv, m, entry); + if (a.data_exceeded) { + agent_report_tail(&a); + if (a.report.len > 0) + buf_addstr(&a.report, "\n\n"); + buf_addstr(&a.report, "subagent data bound exceeded"); + agent_finish(&a, 1); + } + if (round == AGENT_ITER_MAX) { + agent_conclude(&a, m); + } + } + /* NOTREACHED */ + agent_finish(&a, 1); +} blob - /dev/null blob + dd508fe7044f3bba3ac92299957187d483efccb8 (mode 644) --- /dev/null +++ src/fugu-tool/agent.h @@ -0,0 +1,45 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#ifndef AGENT_H +#define AGENT_H + +#include "buf.h" +#include "proto.h" + +struct worker; +struct tool_ctx; + +/* Launch material is assembled only after the -r worker has reached its + * final pledge/unveil. No credential ever enters these buffers (I15). */ +struct agent_boot { + struct agent_start start; + char model[FUGU_MODEL_ID_MAX + 1]; + char label[FUGU_AGENT_LABEL_MAX + 1]; + struct buf system; + struct buf prompt; + int have_start; +}; + +void agent_boot_init(struct agent_boot *); +void agent_boot_free(struct agent_boot *); +int agent_boot_start(struct agent_boot *, const void *, size_t); +int agent_boot_system(struct agent_boot *, const void *, size_t); +int agent_boot_prompt(struct agent_boot *, const void *, size_t); + +__dead void agent_run(struct worker *, struct tool_ctx *, struct agent_boot *); + +#endif /* AGENT_H */ blob - 50dbf929793defc81d696238ed02245af8e9b516 blob + b06198cd808cda70d20ee3537d1329ccc42fd8ce --- src/fugu-tool/main.c +++ src/fugu-tool/main.c @@ -26,10 +26,13 @@ */ #include +#include +#include #include #include #include +#include #include #include #include @@ -44,6 +47,7 @@ #include "proto.h" #include "job.h" #include "tools.h" +#include "agent.h" static int readonly; /* -r: the subagent role, always read-only */ static volatile sig_atomic_t cancel_tool; @@ -54,6 +58,8 @@ struct tool_state { int have_name; struct buf arg; /* accumulating TOOL_ARG chunks */ int cap; /* max_bg_jobs, from HELLO */ + int agent; /* ephemeral M9 orchestrator */ + struct agent_boot boot; struct event pump_ev; /* periodic drain of background jobs */ }; @@ -128,13 +134,19 @@ unveil_protected(const char *list, size_t len) for (off = 0; off < len; ) { const char *tok = list + off; - size_t toklen = strnlen(tok, len - off); + size_t left = len - off; + size_t toklen = strnlen(tok, left); glob_t g; + int error; + if (toklen == left) + fatalx("fugu-tool: unterminated protect path"); off += toklen + 1; if (toklen == 0) continue; - if (glob(tok, GLOB_NOSORT, NULL, &g) == 0) { + memset(&g, 0, sizeof(g)); + error = glob(tok, GLOB_ERR | GLOB_NOSORT, NULL, &g); + if (error == 0) { size_t k; for (k = 0; k < g.gl_pathc; k++) { @@ -145,7 +157,8 @@ unveil_protected(const char *list, size_t len) } paths[npaths++] = xstrdup(g.gl_pathv[k]); } - } else if (strpbrk(tok, "*?[") == NULL) { + } else if (error == GLOB_NOMATCH && + strpbrk(tok, "*?[") == NULL) { /* a plain path with no matches: block it anyway */ if (npaths == cap) { cap = cap ? cap * 2 : 8; @@ -153,6 +166,10 @@ unveil_protected(const char *list, size_t len) sizeof(*paths)); } paths[npaths++] = xstrdup(tok); + } else if (error != GLOB_NOMATCH) { + globfree(&g); + fatalx("fugu-tool: protect pattern %s: glob error %d", + tok, error); } globfree(&g); } @@ -165,6 +182,55 @@ unveil_protected(const char *list, size_t len) free(paths); } +/* + * A coordinator-expanded snapshot arrives out of band so its size is not tied + * to MAX_IMSGSIZE. Validate the descriptor and declared length, then stream + * NUL-separated literal paths with pread(2). pread is essential: every + * dynamic worker receives a duplicate of the same open file description. + */ +static void +unveil_snapshot(int fd, uint64_t len) +{ + struct stat st; + u_char chunk[4096]; + char path[PATH_MAX]; + uint64_t off = 0; + size_t plen = 0; + ssize_t n; + + if (fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) || st.st_size < 0 || + (uint64_t)st.st_size != len) + fatalx("fugu-tool: malformed protect snapshot descriptor"); + while (off < len) { + size_t want = sizeof(chunk), i; + + if ((uint64_t)want > len - off) + want = (size_t)(len - off); + do { + n = pread(fd, chunk, want, (off_t)off); + } while (n == -1 && errno == EINTR); + if (n <= 0) + fatalx("fugu-tool: truncated protect snapshot"); + for (i = 0; i < (size_t)n; i++) { + if (chunk[i] == '\0') { + if (plen == 0) + fatalx("fugu-tool: empty protect snapshot path"); + path[plen] = '\0'; + if (unveil(path, "") == -1) + fatal("unveil protect %s", path); + plen = 0; + } else { + if (plen >= sizeof(path) - 1) + fatalx("fugu-tool: protect snapshot path too long"); + path[plen++] = (char)chunk[i]; + } + } + off += (uint64_t)n; + } + if (plen != 0) + fatalx("fugu-tool: unterminated protect snapshot path"); +} + static int tool_lockdown(struct worker *w, struct imsg *imsg) { @@ -172,22 +238,48 @@ tool_lockdown(struct worker *w, struct imsg *imsg) struct hello_tool h; const u_char *p; size_t len; - int write_ok, net_ok; + int passed_fd, snapshot_fd, write_ok, net_ok; p = imsg_bytes(imsg, &len); if (p == NULL || len < sizeof(h)) fatalx("fugu-tool: short HELLO"); memcpy(&h, p, sizeof(h)); + passed_fd = imsg_get_fd(imsg); + if (passed_fd != -1) { + close(passed_fd); + fatalx("fugu-tool: unexpected HELLO descriptor"); + } + if ((h.allow_write != 0 && h.allow_write != 1) || + (h.allow_subprocess_net != 0 && h.allow_subprocess_net != 1) || + (h.agent != 0 && h.agent != 1) || + (h.protect_snapshot != 0 && h.protect_snapshot != 1) || + h.max_bg_jobs < 0) + fatalx("fugu-tool: malformed HELLO policy"); + if (h.agent && !readonly) + fatalx("fugu-tool: agent mode requires -r"); + ts->agent = h.agent; /* the subagent role is read-only regardless of the gate (I15) */ - write_ok = !readonly && h.allow_write; - net_ok = !readonly && h.allow_subprocess_net; - ts->cap = h.max_bg_jobs; + write_ok = !readonly && !h.agent && h.allow_write; + net_ok = !readonly && !h.agent && h.allow_subprocess_net; + ts->cap = h.agent ? 0 : h.max_bg_jobs; /* protect list first, while the whole tree is still readable */ - if (len > sizeof(h)) - unveil_protected((const char *)p + sizeof(h), - len - sizeof(h)); + if (h.protect_snapshot) { + if (len != sizeof(h)) + fatalx("fugu-tool: malformed protect snapshot HELLO"); + snapshot_fd = h.protect_len > 0 ? FUGU_PROTECT_FD : -1; + if (snapshot_fd != -1) { + unveil_snapshot(snapshot_fd, h.protect_len); + close(snapshot_fd); + } + } else { + if (h.protect_len != 0) + fatalx("fugu-tool: unexpected protect snapshot length"); + if (len > sizeof(h)) + unveil_protected((const char *)p + sizeof(h), + len - sizeof(h)); + } /* * The project tree (read/write/create per the gate), then the @@ -235,8 +327,13 @@ tool_lockdown(struct worker *w, struct imsg *imsg) } else { char exec_p[256]; - strlcpy(exec_p, "stdio rpath getpw proc exec prot_exec tty", - sizeof(exec_p)); + /* OpenBSD ksh requests these promises during its own startup, + * even for an inspection-only command. The worker itself keeps + * no write/create promises, and the inherited project unveil is + * strictly rx, so subprocesses still have no effective mutation + * authority. */ + strlcpy(exec_p, "stdio rpath wpath cpath fattr chown flock " + "getpw proc exec prot_exec tty", sizeof(exec_p)); if (net_ok) strlcat(exec_p, " inet dns", sizeof(exec_p)); if (pledge("stdio rpath proc exec", exec_p) == -1) @@ -306,6 +403,26 @@ tool_dispatch(struct worker *w, struct imsg *imsg) size_t len; switch (imsg_get_type(imsg)) { + case FUGU_IMSG_AGENT_START: + p = imsg_bytes(imsg, &len); + if (!ts->agent || agent_boot_start(&ts->boot, p, len) == -1) + fatalx("fugu-tool: bad AGENT_START"); + return (0); + case FUGU_IMSG_AGENT_SYSTEM: + p = imsg_bytes(imsg, &len); + if (!ts->agent || agent_boot_system(&ts->boot, p, len) == -1) + fatalx("fugu-tool: bad AGENT_SYSTEM"); + return (0); + case FUGU_IMSG_AGENT_PROMPT: + p = imsg_bytes(imsg, &len); + if (!ts->agent || agent_boot_prompt(&ts->boot, p, len) == -1) + fatalx("fugu-tool: bad AGENT_PROMPT"); + return (0); + case FUGU_IMSG_AGENT_RUN: + if (!ts->agent || !ts->boot.have_start || imsg_get_len(imsg) != 0) + fatalx("fugu-tool: bad AGENT_RUN"); + agent_run(w, ts->ctx, &ts->boot); + /* NOTREACHED */ case FUGU_IMSG_TOOL_TURN_BEGIN: case FUGU_IMSG_TOOL_TURN_COMMIT: case FUGU_IMSG_TOOL_TURN_ABORT: { @@ -444,6 +561,7 @@ main(int argc, char *argv[]) fatal("sigaction SIGTERM"); buf_init(&ts.arg); + agent_boot_init(&ts.boot); /* * Transitional pledge. It is a superset of the steady promises; blob - 50651042344b4069bd7401927d8686323b97114e blob + 89ee46cf13ae45e6df3c31d7ebf8c852d91dace5 --- src/fugu-tool/tools.c +++ src/fugu-tool/tools.c @@ -214,7 +214,10 @@ tool_read(struct tool_ctx *ctx, const struct json *j, u_char chunk[65536]; int64_t offset = 1, limit = -1, off_v, lim_v; int64_t cur_line = 1, total_lines, win_hi, next_off = 0; - int fd, t, why, truncated = 0, plain, had_bytes = 0; + int64_t truncated_line = 0; + size_t line_bytes = 0; + int fd, t, why, truncated = 0, oversized_line = 0; + int plain, had_bytes = 0; ssize_t n; u_char last = '\n'; @@ -289,16 +292,25 @@ tool_read(struct tool_ctx *ctx, const struct json *j, had_bytes = 1; for (i = 0; i < n; i++) { + /* Saturate after the only threshold we need to test. */ + if (line_bytes <= READ_BYTE_CAP) + line_bytes++; if (cur_line >= offset && cur_line <= win_hi) { if (out->len < READ_BYTE_CAP) buf_addc(out, chunk[i]); else if (!truncated) { truncated = 1; next_off = cur_line; + truncated_line = cur_line; } } - if (chunk[i] == '\n') + if (truncated_line == cur_line && + line_bytes > READ_BYTE_CAP) + oversized_line = 1; + if (chunk[i] == '\n') { cur_line++; + line_bytes = 0; + } } last = chunk[n - 1]; } @@ -324,9 +336,17 @@ tool_read(struct tool_ctx *ctx, const struct json *j, fresh_record(ctx->fresh, path, &st); - if (!truncated && win_hi < total_lines) + if (oversized_line) { + /* A line offset cannot resume within an over-cap line. */ + next_off = truncated_line + 1; + buf_addf(out, "\n[fugu: %lld lines total; window truncated " + "at 1 MiB on line %lld; remainder omitted; next offset " + "%lld%s]\n", (long long)total_lines, + (long long)truncated_line, (long long)next_off, + next_off > total_lines ? " (end-of-file)" : ""); + } else if (!truncated && win_hi < total_lines) next_off = win_hi + 1; - if (truncated || next_off != 0) + if (!oversized_line && (truncated || next_off != 0)) buf_addf(out, "\n[fugu: %lld lines total; re-read from " "offset %lld for more%s]\n", (long long)total_lines, (long long)next_off, truncated ? " (window truncated " blob - 7cf7cc0631c7dd831982912231de8c3a334e1262 blob + 28f63779f1fa42debfdf0de1872100134d910011 --- src/fugu-tty/ui.c +++ src/fugu-tty/ui.c @@ -86,6 +86,7 @@ struct sess { /* a model-picker entry */ struct pick { int slot; + int64_t context_window; char name[64]; char id[FUGU_MODEL_ID_MAX + 1]; }; @@ -1158,6 +1159,7 @@ pick_key(struct ui *u, enum ed_key k, uint32_t cp) m = &u->models[match[u->pick_sel]]; memset(&sel, 0, sizeof(sel)); sel.slot = m->slot; + sel.context_window = m->context_window; buf_init(&b); buf_add(&b, &sel, sizeof(sel)); buf_addstr(&b, m->id); @@ -1635,6 +1637,7 @@ ui_render(struct worker *w, struct imsg *imsg) tl = sizeof(m->id) - 1; memcpy(m->id, nul + 1, tl); m->slot = me.slot; + m->context_window = me.context_window; break; } case FUGU_IMSG_UI_MODELS_END: