commit 6b55003318869d06a4f7b148d2bf0649d22053ac from: Isaac date: Fri Jun 26 22:30:06 2026 UTC Import fugu: OpenBSD-native privsep AI coding agent Five pledge/unveil-sandboxed binaries (fugu parent + fugu-{ui,net,exec,fetch}), base system + vendored jsmn.h only. Multi-provider net worker (Anthropic + OpenAI-compatible wire protocols), vi-modal curses TUI with markdown rendering, runtime /model picker, SSRF-guarded fetch worker, sandboxed exec tools, and a user/model-invocable skills subsystem. Includes the regress suites, man pages, ADRs, and the .scratch issue tracker. commit - /dev/null commit + 6b55003318869d06a4f7b148d2bf0649d22053ac blob - /dev/null blob + b7432a22b56a53536c4f3e1a6774a14d76af77d4 (mode 644) --- /dev/null +++ .gitignore @@ -0,0 +1,26 @@ +# Build artifacts (objects + depfiles live in-tree next to sources) +*.o +*.d +*.core + +# Per-subsystem build output dirs (regress uses obj/) +obj/ + +# Generated parser (yacc from common/parse.y) +/src/parent/parse.c +/src/parent/parse.h +/regress/parse/parse.c +/regress/parse/parse.h + +# Built worker binaries +/src/parent/fugu +/src/ui/fugu-ui +/src/net/fugu-net +/src/exec/fugu-exec +/src/fetch/fugu-fetch + +# Regress test binaries +regress/*/*_test + +# Release tarball staging (make dist) +/fugu-*.tar.gz blob - /dev/null blob + 1c2ac7eccd5ba9586a9fc21a7cc462c1c3e0b78e (mode 644) --- /dev/null +++ .scratch/api-hardening/README.md @@ -0,0 +1,179 @@ +# API hardening sweep: kill the length/NUL bug class at the contract level + +Status: done (sweep landed 2026-06-26; deferred items below) +Origin: audit/fix session 2026-06-26 — `git log --grep audit` and the workflow output at `~/.claude/.../tasks/w3ojf5irm.output` for the history. + +## Outcome (2026-06-26) + +All six sub-tasks landed. Regress total grew from 660 to 687 checks across 21 suites, 0 failures. + +- **1 (buf contract):** `buf_puts` → `buf_puts_cstr` (81 call sites in 19 files); added `buf_terminate` + `buf_cstr` (assert-on-default-build); migrated every `buf_putc(x, '\0'); x.len--;` site (28+ sites across src + regress); new `regress/buf/` suite with 23 checks covering terminate/cstr/embedded-NUL round-trip. +- **2 (obj_getstr_n promote):** added as public `obj_getstr_n` in `src/common/json.{c,h}`; the static copy in `fetch_run.c` removed; fetch_run.c now uses the common one. The wider migration of `obj_getstr` callers in `openai.c` / `anthropic.c` / `exec_run.c` / `skills.c` was AUDITED but most sites are dispatch-only (strcmp on enum-like strings) or feed `json_kv_string` (itself strlen-based — a separate migration). See "Deferred" below. +- **3 (json_tok_strdup doc):** header comment in `json.h` steers wire/storage callers to `_n`. +- **4 (tlsconn audit):** `tls_error()` is called only inside `tlsconn_dup_error`; the file-level contract is now documented inline. No new tls_error leaks. +- **5 (ACL tests):** four host-match boundary checks added to `regress/fetch/fetch_test.c` — exact-pattern rejects "evillocalhost" vs "localhost"; dot-suffix accepts "localhost" vs ".localhost"; dot-suffix rejects "evillocalhost" vs ".localhost" (the label-anchor regression). +- **6 (secret_load):** new `secret_load(have_inline, val, have_file, path, &out, &len)` in `parent.c`; `netprov_key` returns the true byte length; `delegate_net` and the Kagi path in `delegate_fetch` now `freezero(buf, keylen)` instead of `strlen` — so a file-loaded key with embedded NUL bytes is wiped completely. Old `resolve_secret` kept as a thin back-compat wrapper. + +## Deferred + +- Wider `obj_getstr` → `obj_getstr_n` migration in `exec_run.c` (paths/commands flowing into `open(2)`, `argv`), `openai.c` (tool name/description into `json_kv_string`), and `anthropic.c` (error message). Most sites need both `_n` AND a length-aware sink (`json_kv_string_n` etc.) — a separate sub-sweep. +- Migration of remaining `obj_getstr` callers whose sinks are themselves strlen-based (`json_kv_string`, `buf_puts_cstr`). The producer fix has marginal value without the sink fix. +- NUL-validation guards at the security boundary (exec_run.c paths/commands): explicit `if (strlen(s) != s_len) reject;` after `obj_getstr_n`. Not a behavior regression today (the strlen sink truncates uniformly), but the right defense. + +## Goal + +The audit and fix session closed a class of correctness/safety bugs caused by +mixing length-aware producers with strlen-based consumers (the canonical case +was `on_tool_call` in `net_api.c` reading past `buf->len` via strlen, splicing +heap garbage into outbound JSON; siblings included `openai.c` accumulating +streamed tool args via `buf_puts`, and `fetch_run.c` using `strlen` for HTTP +body length on `json_tok_strdup` output that can carry embedded NULs). + +This issue tightens the **API contracts** so that future code physically +cannot reintroduce the same shape. Six sub-tasks below; do them in order or +land them piecewise — none blocks another beyond what's noted. + +## What's already done (do not redo) + +- `json_tok_strdup_n(d, tok, &lenp)` exists in `src/common/json.{c,h}` and is + used by the openai, anthropic, and fetch_run fixes. +- An ad-hoc `obj_getstr_n` is **static** inside `src/fetch/fetch_run.c` — its + shape is the prototype for the promoted version (see task 2). +- `acl_path_prefix_match(path, pref, preflen)` is a **static** helper in + `src/fetch/fetch_run.c` (around line 1054, see commit) — promote/extend it + in task 5. +- `tlsconn_connect`'s `*errstr` already takes ownership (caller `free()`s) — + task 4 is about encapsulating libtls error access more cleanly, not the + ownership change itself. + +## Sub-tasks + +### 1. `struct buf` contract: make NUL-termination an explicit, asserted invariant + +The bug we hit was that `b->data[b->len]` can be uninitialized after +`buf_puts`. The post-fix convention is "callers explicitly NUL-terminate +when they later read `.data` as a C string." Bake that into the API. + +Concrete changes in `src/common/buf.{c,h}`: + +- Rename `buf_puts(b, s)` → `buf_puts_cstr(b, s)` so its strlen semantics are + loud at every call site. Mechanical sed-then-review. +- Add `void buf_terminate(struct buf *b)` that does + `buf_putc(b, '\0'); b->len--;` (the pattern fugu now uses everywhere). Use + it at `on_tool_call`, `convo_add_user`, etc. for symmetry. +- Add a debug-only helper: + `static inline const char *buf_cstr(const struct buf *b)` that + `assert(b->data[b->len] == '\0')` then returns `b->data`. Migrate the + `json_raw(&w, b.data)` callers to `json_raw(&w, buf_cstr(&b))`. Under + `FUGU_DEBUG=1` this would have caught the original overrun at first hit. + +Verification: `make FUGU_DEBUG=1 regress` still passes (existing tests +already exercise the converted call sites); `make` (default) is unchanged. + +### 2. Promote `obj_getstr_n` to a shared primitive + +Currently a static helper in `fetch_run.c`. Many callers across `openai.c`, +`anthropic.c`, `net_run.c`, `parent.c`, `confload.c`, etc. still use the +strlen-based `obj_getstr` (or hand-rolled equivalents) on model/user input +that can carry embedded NULs. + +- Add `char *obj_getstr_n(const struct json_doc *d, int obj, const char *key, + size_t *lenp)` to `src/common/json.{c,h}` (alongside `json_tok_strdup_n`). +- Grep `obj_getstr(` and audit each call site: + - if the caller hands the string to a strlen-based writer that crosses the + network or storage boundary → migrate to `obj_getstr_n` + - if the caller only uses the string for equality / dispatch (`strcmp` on + a small enum) → leave it (NUL semantics are equivalent) +- Specifically check: `src/common/anthropic.c` (audit flagged it as + "probably has the same pattern but wasn't fully read" — confirm or fix); + `src/exec/exec_run.c` (paths and shell commands — NULs here are + exploitable on the exec side, treat carefully). + +### 3. Deprecate `json_tok_strdup` once callers are migrated + +After task 2 completes, the only remaining strlen-based callers should be +ones that legitimately don't care about embedded NULs. Document this: + +- Add a comment to `json_tok_strdup` in `json.h` saying "prefer + `json_tok_strdup_n` when the decoded bytes flow into a wire format or + storage; this strlen-truncating variant is for dispatch/equality only." +- Optional: add `__attribute__((__deprecated__(...)))` if you want compile- + time pressure to migrate. Probably overkill given the small caller list. + +### 4. `tlsconn`: confine libtls error pointer to the module + +`tlsconn_connect` already returns an owned `*errstr`. But the underlying +borrow-then-free pattern is intrinsic to libtls and could resurface in any +new `tlsconn_*` function (e.g. a future `tlsconn_renegotiate`, or if someone +exposes `tlsconn_get_state`). Encapsulate once: + +- Add `static char *tlsconn_dup_error(struct tls *ctx)` in `tlsconn.c` — + already added by the fix workflow; promote it to a small documented + helper inside the file (keep it static — it's an internal contract). +- Audit `tlsconn.c`: any function that captures a `tls_error()` pointer for + return must go through `tlsconn_dup_error` BEFORE the next `tls_free` or + `tls_close`. The fix workflow handled `tlsconn_connect`'s three call sites + + the early `tls_client failed` literal; verify nothing else has crept in. + +### 5. ACL: route every path/host check through one helper + +`fetch_run.c:acl_path_prefix_match` exists. Sister checks (host match, +deny-list, future allow-list extensions) should all go through similar +named helpers so the segment-boundary rule, case-folding, and any future +normalization live in ONE place: + +- Add `static int acl_host_match(const char *host, const char *pat)` if not + already factored — currently `acl_host_match` inlines the dot-suffix + logic. Trust but verify it's not vulnerable to the same off-by-one + pattern (audit said it isn't, but a unit test under + `regress/fetch/fetch_test.c` codifying the rule is worth having). +- Consider normalizing patterns at config-load time: strip trailing slashes + on path patterns, lowercase host patterns. Push the normalization into + `confload.c` so runtime checks are simpler. + +### 6. `resolve_secret`: length-aware secret loading + +Current `resolve_secret` in `src/parent/parent.c` returns a `char *` and +`explicit_bzero` happens at strlen boundary in callers. For keys with +embedded NULs (rare but legitimate for raw-bytes credentials) the trailing +bytes leak past `bzero`. + +- Add `int secret_load(const char *path, char **out, size_t *outlen)` that + returns (ptr, len) so callers can `freezero(buf, len)` correctly. +- Use it in `delegate_net` / `delegate_fetch`. The M_SECRET payload already + carries a length over imsg, so net/fetch end is fine. +- Keep `resolve_secret` as a thin wrapper around `secret_load` until all + callers migrate, then remove. + +## Constraints + +- **OpenBSD KNF** is the tiebreaker on style (the user has this as a + feedback memory: prefer base-system idioms when in doubt). +- **No new dependencies.** Stay within the existing toolchain. +- **Privsep contract is sacred.** No new path between workers; no new + imsg variant unless the existing M_TOOL_* / M_CONFIG / M_SECRET set + cannot model the change. +- **Test-first where the cost is low.** Each task should land with a + regress test in the appropriate `regress//` dir. The audit + fix workflow's tests are the template. + +## Verification + +- `make` clean (no warnings, default flags) +- `make FUGU_DEBUG=1` clean (UBSan-trap + Fortify + -Wextra -Wformat=2) +- `make -C regress regress` — all dirs pass, total check count ≥ 870 +- A focused regress test exists for each migration that introduces a new + invariant (e.g. task 1 should add a `regress/buf/` if not present and + test that `buf_terminate` + `buf_cstr` round-trips) +- The audit summary in the session output (`w5m6qp6ti.output`) should + re-run with zero confirmed findings against the modified files + +## Out of scope + +- Adding ASan/MSan support. OpenBSD's clang ships no runtime; UBSan-trap is + what's portable. Don't try to graft an ASan port. +- Refactoring providers (anthropic.c, openai.c) beyond the embedded-NUL + migration. The agent-loop/streaming design is sound. +- The skills feature. Skills landed in two prior phases; the contract is + stable. Hands off unless a task above touches a function skills uses, + in which case re-run the skills regress (`regress/skills/`). blob - /dev/null blob + 4636cfdab272ae9b5379d2c75cbd02651695bfb9 (mode 644) --- /dev/null +++ .scratch/fetch-tls-refactor/README.md @@ -0,0 +1,119 @@ +# fetch_run.c: route TLS through `tlsconn` instead of open-coding libtls + +Status: done (2026-06-26) +Origin: /code-review ultra finding #12, 2026-06-26 session. + +## Outcome (2026-06-26) + +http_fetch now attaches TLS via `tlsconn_attach` instead of open-coding `tls_client` / `tls_configure` / `tls_connect_socket` / `tlsconn_do_handshake`. Real libtls error reasons now surface to the user (e.g. "TLS handshake failed: certificate verification failed: certificate has expired" instead of the historical canned literal "TLS handshake failed"). All four `tlsconn_*` hardening properties — owned `*errstr` strings, the `tlsconn_dup_error` chokepoint, the centralised handshake retry loop, and the lifecycle invariants under heap churn — now protect fetch the same way they protect net. + +Total regress 705 → 710 (0 failures), +5 from the new `tlsconn_attach` test. + +### New tlsconn primitive + +`tlsconn_attach(struct tlsconn *, struct tls_config *, int fd, const char *servername, char **errstr)` complements `tlsconn_connect`. fetch needs it because its `dial()` does the TCP connect under its own SSRF guard + wall-clock deadline; `tlsconn_connect` (which dials internally via libtls) would silently bypass both. Ownership of `fd` transfers to the `tlsconn` on success; on failure `c.fd` stays -1 and the caller still owns the fd. + +### fetch_run.c changes + +- `http_fetch` signature: `const char **errstr` → `char **errstr` (caller-owned heap string). +- `dial` signature: same flip — `*errstr` is heap-allocated at every failure site. +- All `*errstr = "literal"` writes migrated to `*errstr = xstrdup("literal")`. +- TLS section reduced to: `if (tlsconn_attach(&c, fs->tlscfg, fd, host, &tlserr) == -1) { compose; free(tlserr); goto out; }`. +- `out:` label uses `tlsconn_close(&c)` (idempotent; handles the never-attached case too). +- Three callers (`do_web_search`, `do_web_fetch`, `do_http_request`) updated to `char *err = NULL` and `free(err)` after the buf_printf. + +### New regress + +`test_attach_caller_keeps_fd_on_failure` in `regress/tlsconn/`: caller dials a stub that immediately closes; `tlsconn_attach` returns -1 with heap-owned `*errstr` (heap-stable under churn); `c.fd` stays -1; caller's fd is still its responsibility to close. Codifies the asymmetric-ownership invariant between `tlsconn_connect` and `tlsconn_attach`. + +### Verified + +- `make` clean +- `make FUGU_DEBUG=1` clean (UBSan-trap + Fortify + Wextra + Wformat=2) +- `make regress` — 710 checks, 0 failures across all 22 suites +- Adversarial workflow review: 3 lenses (correctness / memory-leaks / KNF), all `verdict: ok`. Cosmetic fixes from the review applied. + +### Live verification (deferred) + +The handoff's Verification section calls for a hit against an expired-cert host to see the new error text. Defer until the user reinstalls the port (`/usr/local/bin/fugu` is still on the previous build). Suggested live test once reinstalled: + +``` +/web_fetch https://expired.badssl.com/ +``` + +Before: `web_fetch: expired.badssl.com: TLS handshake failed` +After: `web_fetch: expired.badssl.com: TLS handshake failed: certificate verification failed: certificate has expired` + +## Goal + +`http_fetch` in `src/fetch/fetch_run.c` (around line 822) calls `tls_client`, `tls_configure`, `connect_socket`, `tls_handshake` directly, instead of going through the `tlsconn` module that net_api uses. Consequences: + +1. Every TLS error in the fetch worker surfaces a canned string literal (`"TLS handshake failed"`, etc.) instead of the real `tls_error(c.ctx)` reason. Users can't tell a cert verify failure from a hostname mismatch from a connect-time DNS error. +2. Any future hardening of `tlsconn_connect` — SNI policy, cert pinning, OCSP, the recent owned-errstr ownership change — silently bypasses fetch. +3. The use-after-free bug class the audit fixed in `tlsconn_connect` could re-emerge here: `tls_error()` returns a borrowed pointer; if fetch_run captures it before `tls_close()`/`tls_free()`, it has the same lifetime hazard. + +Route `http_fetch` through `tlsconn` so the contract is enforced in one place. + +## Why this matters + +Today the fetch worker handles: + +- Kagi search (`web_search`) +- Web page fetch with HTML→text rendering (`web_fetch`) +- General HTTP request to allowlisted hosts (`http_request`) + +All three share the `http_fetch` helper. All three see degraded error messages on TLS failure and don't benefit from any tlsconn hardening. + +This is pre-existing (not a regression from this session), but: the audit hardened `tlsconn_connect`'s `*errstr` ownership; with fetch open-coding TLS, that hardening only protects net, not fetch. Anyone touching fetch's TLS path has to remember to keep it in sync with tlsconn. + +## What's already done + +- `tlsconn_connect` returns an owned, heap-allocated `*errstr` (`char **`, caller frees) — landed in the audit-fix workflow. +- `tlsconn.c` has `tlsconn_dup_error` (static helper) centralising the libtls error capture. +- API hardening handoff documents the broader pattern (`.scratch/api-hardening/README.md` task 4). + +This handoff is the concrete refactor that picks up that work. + +## Approach + +Migrate `http_fetch` (and any sister helpers in fetch_run.c that call libtls directly) to: + +1. `tlsconn_client_config(cafile)` for the cert config (already shared) +2. `tlsconn_connect(&c, cfg, host, port, &err)` for connect+handshake +3. `tlsconn_write_all(&c, ...)` for the request +4. `tlsconn_read(&c, ...)` in the response loop +5. `tlsconn_close(&c)` on cleanup + +Match the shape `net_api.c:net_api_call` uses (lines 169-180). On `tlsconn_connect` returning -1, free the owned `err` string after emitting it. + +The CA bundle path lives in netstate as `cafile` and in fetchstate as a parallel field — make sure fetch's tls_config equivalent uses the same path semantics. + +## Concrete starting points + +- `src/fetch/fetch_run.c:822` — start of `http_fetch`, the function to migrate +- `src/fetch/fetch_run.c` — search for `tls_client`, `tls_configure`, `tls_handshake`, `tls_close`, `tls_free` to find all sites +- `src/common/tlsconn.{c,h}` — the API to use +- `src/net/net_api.c:127-232` — `net_api_call` is the working example; mirror its idiom +- `src/fetch/fetch.c` and `src/fetch/fetch_run.h` — fetchstate definition; check whether `cafile` and `tlscfg` are already plumbed (similar to net's `netstate.tlscfg`) + +## Constraints + +- OpenBSD KNF +- Preserve the existing `FETCH_MAXBODY` bound on response size +- Preserve pledge promises: fetch worker pledges `stdio inet dns`; tlsconn calls don't open files (the CA bundle is loaded before pledge tightens — confirm) +- Don't introduce a new imsg variant; the existing M_TOOL_REQUEST/M_TOOL_RESULT/M_ERROR set is sufficient +- The error message format that bubbles up to M_ERROR should now include the real libtls reason — update the `net_emit_error` calls accordingly so the user-facing diagnostic is informative + +## Verification + +- `make` and `make FUGU_DEBUG=1` clean +- `make regress` — fetch tests pass (currently the harness doesn't inject TLS failures; see the `tls-regress-harness` handoff) +- Manual test against a host with an expired or mismatched cert: confirm the M_ERROR text now contains a libtls-quality reason rather than the generic literal +- After the `tls-regress-harness` work lands, add a fetch-side test that injects a TLS failure and asserts the M_ERROR carries the real reason +- Live: run `/web_fetch https://expired.badssl.com` (or equivalent) and check the user sees a useful error + +## Out of scope + +- Replacing the HTTP layer in fetch with the one in `src/common/http.c` (net's). They serve different purposes (fetch streams body for HTML→text conversion); leave the HTTP framing alone, change only the TLS plumbing. +- Refactoring the Kagi-specific code path. Kagi is just a host on top of `http_fetch`; the migration helps it transparently. +- Adding HSTS/cert-pinning policy. That belongs in `tlsconn_client_config` and lives in the API hardening sweep. blob - /dev/null blob + 5008bfc0ddcb51cebdf6ed3e7d44a3aa54b88d41 (mode 644) --- /dev/null +++ .scratch/journal-turn-txn/README.md @@ -0,0 +1,89 @@ +# Journal transactions: failed turns must not leave orphan records + +Status: ready-for-agent +Origin: /code-review ultra finding #11, 2026-06-26 session. + +## Goal + +When a turn fails mid-flight (HTTP error from the provider, network drop, malformed stream), the in-memory convo is truncated back to a mark — but the JOURNAL already contains the records the turn wrote BEFORE submitting (skill marker, synth assistant tool_use, synth user tool_result, and any partial assistant message blocks). On resume those records replay, putting the convo in a state the user never actually experienced and the model never actually responded to. + +This is most visible with the Phase 2 skills synth pair (three records in flight), but the same shape applies to any future "write-then-submit" turn. + +Add transactional boundaries to the journal so resume can faithfully reproduce ONLY the turns that succeeded. + +## Why this matters + +Today after a failure mid-skill: + +- In-memory convo: rolled back to before the synth pair. +- Journal: `[marker, assistant(tool_use), user(tool_result)]` — orphan trio. +- Next turn the user types "Hello", journal appends a normal user message. +- Resume replay: the orphan trio gets converted back to convo state, then the user("Hello") follows. +- The next API request goes out with `[…, assistant(tool_use), user(tool_result), user("Hello")]` — two adjacent user messages, which Anthropic Messages strict-rejects, and OpenAI tolerates but treats as confused context. + +Skills magnify this because Phase 2 records three messages per invocation; pre-Phase-2 a failed turn left a single orphan user message that mostly didn't break alternation. + +## What's already done + +- The synth pair shape (skills_synth_invocation produces `user(typed) + assistant(tool_use) + user(tool_result)`) — landed in the review-fix workflow. +- convo_replay now renders tool_result blocks — same workflow. + +The orphan-on-failure case was deferred from that workflow because it needs a journal protocol change, not a localised fix. + +## Approach + +Add a `{"fugu":"turn_failed"}` marker to the journal record vocabulary. On failure, emit the marker right after the partial turn's records. On resume, when the replay walker sees `turn_failed`, scan backwards through journal records until it hits the most recent transactional boundary (a skill marker, a `/clear` marker, a `/compact` marker, or the start of the journal) and DROP all records in between. + +This treats every skill marker and every regular user-message boundary as an implicit "begin transaction"; turn_failed is the abort signal. + +### Concrete shape + +In `src/ui/convo.c`: + +- Add `RK_TURN_FAILED` to `enum rec_kind`. +- Add `convo_marker_turn_failed(struct buf *out)` to emit `{"fugu":"turn_failed"}`. +- In `record_kind`, recognise the new marker. +- In `resume_line`, when seeing `RK_TURN_FAILED`: + - Walk backwards through `rc->cv` removing the last K messages (free them) up to and including the marker that opened the failed transaction. The marker is the closest previous record matching `RK_SKILL` OR a "user typed" boundary. **Note:** since record_kind only sees one line at a time, you'll need a richer replay state — track the convo-position-at-last-boundary in `struct resume_ctx`, and on TURN_FAILED truncate to that saved position. +- After the truncation, the next non-marker record reopens a new transaction. + +In `src/ui/ui.c` and `src/ui/ui_curses.c`: + +- After a failed turn (the `r == 1` branch where we already do `convo_truncate(cv, mark)`), also append a `turn_failed` marker to the journal. + +In `src/ui/convo.h` (the header for the new functions). + +### Wire format + +Use `{"fugu":"turn_failed"}` — symmetric with `/clear` and `/compact`. No payload needed since the marker semantically means "undo from the previous boundary." + +## What's been ruled out + +- **"Just don't journal until success."** The streamed assistant message records arrive INTERLEAVED with the turn; they must journal eagerly or you lose them on crash. Buffering the whole turn defeats the append-only-as-it-happens design. +- **"Reorder so the response comes BEFORE the synth pair."** Can't — the response is generated from those messages. +- **"Make the synth pair atomic in the imsg protocol."** Wrong layer; this is a journal/resume concern, not an imsg concern. + +## Concrete starting points + +- `src/ui/convo.c:241` — `enum rec_kind` definition +- `src/ui/convo.c:84-107` — pattern for marker emitters (`convo_marker_clear`, `convo_marker_compact`) +- `src/ui/convo.c:278-310` — `resume_line` switch where the new case lands +- `src/ui/ui.c:373-381` — the `r == 1` failure branch where `convo_truncate(cv, mark)` runs today +- `src/ui/ui_curses.c` — equivalent path inside the M_TURN_DONE / M_ERROR handlers + +## Constraints + +- OpenBSD KNF style +- Backward compatibility: old journals (no turn_failed markers, possibly with orphans) must still replay without crashing — just produce the buggy convo they always produced. Don't auto-clean retroactively. +- Resume's CP_INFO display: emit something like "previous failed turn discarded" so the user sees what happened. + +## Verification + +- A new regress test under `regress/resume/`: write a journal with marker + synth pair + turn_failed, then a fresh user message. Replay. Assert the convo contains only the fresh user message, and a CP_INFO was emitted. +- Live: invoke `/smoke` against a deliberately broken provider (e.g. temporarily set an invalid API key), watch fugu error, then exit, then `fugu -c` to resume. The resumed convo should NOT contain the failed skill invocation. +- `make regress` total count rises by the new tests; all dirs green. + +## Out of scope + +- A general transaction API for arbitrary multi-record sequences. This is a focused fix for the skill-synth + failed-turn case. +- Crash recovery (process killed mid-write). The journal is append-only with O_APPEND; partial lines are handled by the existing parser. blob - /dev/null blob + 384391355dd860b0b922abe4da40ccc8b1c1cefc (mode 644) --- /dev/null +++ .scratch/tls-regress-harness/README.md @@ -0,0 +1,194 @@ +# TLS regress harness: failure injection + use-after-free regression test + +Status: done (2026-06-26) +Origin: audit session 2026-06-26 — the tlsconn use-after-free fix landed +without a regress test because the existing harness only exercises the +happy path. See `src/common/tlsconn.c:tlsconn_connect` (already fixed) and +`regress/net/net_test.c` for the existing stub TLS server. + +## Outcome (2026-06-26) + +New `regress/tlsconn/` suite. 18 checks, all passing against the fixed code. Total regress 687 → 705 (0 failures across 22 suites). + +Tests landed: + +- **Mode 3 (connect-to-nothing):** `tlsconn_connect` against an unbound port. Asserts `*errstr` is non-NULL, non-empty, length stable across heap churn, byte-stable across heap churn, and that `c.fd == -1` / `c.ctx == NULL` post-failure. +- **Mode 2 (TCP-accept-then-close):** fork()ed stub binds 127.0.0.1:0, accepts, immediately closes. tls_handshake fails inside libtls; same heap-churn assertions on the captured error string. +- **close-on-uninitialised:** `tlsconn_close` on a memset-zeroed `struct tlsconn` is a clean no-op and idempotent. +- **repeated-failures-distinct-pointers:** Two back-to-back `tlsconn_connect` failures produce two errstr allocations with distinct heap pointers but identical message bytes — rules out a future regression that returns a shared static buffer. + +Heap churn primitive: 32 rounds × 64KB allocations, each filled with `0xA5` then freed, forcing OpenBSD's default malloc.conf junk-fill (`0xDB`) to overwrite any region a stale `tls_error()` pointer might have referenced. + +## UAF regression validation (the proof) + +The handoff asked for a revert-the-fix experiment to prove the test catches the original bug. Done: + +- Replaced the three `tlsconn_dup_error(c->ctx)` calls in `tlsconn_connect`'s fail paths with the pre-fix borrowed-pointer assignment `*errstr = (char *)tls_error(c->ctx)`. +- Rebuilt + ran `make regress` → process aborted with **SIGABRT** (from a CHECK macro tripping an internal libc consistency check on the corrupted heap region). +- Restored the fix → 18 checks pass cleanly. + +The test cleanly distinguishes the fixed implementation from the buggy one. The exact failure mode (SIGABRT vs CHECK-FAIL vs hang) varies between runs because OpenBSD's heap-junk pattern depends on free/alloc order — but the test ALWAYS detects abnormal behavior with the UAF in place. + +## What's already done + +- The use-after-free in `tlsconn_connect` is fixed: `*errstr` is now + `xstrdup`-allocated and caller-owned. Signature in `tlsconn.h` is + `char **errstr` (not `const char **`); contract documented in the + header comment around line 39. +- Callers in `src/net/net_api.c` (lines ~169 and ~273) `free(err)` after + `net_emit_error` consumes it. +- A static `tlsconn_dup_error(struct tls *ctx)` helper in `tlsconn.c` + centralises the libtls error capture pattern. See API hardening issue + task 4 for completing that encapsulation. + +## Goal + +Add failure injection to the regress harness for libtls failure modes, +then write a focused regression test for the use-after-free that was just +fixed in `tlsconn_connect`. The test should fail loudly against the +pre-fix code (read-after-free of the libtls error buffer) and pass on the +post-fix code. + +The harness should be reusable: any future TLS robustness work — cert +rotation, peer verify errors, partial handshakes — can lean on the same +injection primitives. + +## Why this matters + +The use-after-free at `tlsconn.c:70` (pre-fix) was confirmed real by three +adversarial verifiers but stayed latent because: + +- OpenBSD's default `malloc.conf` junk-fills freed memory, so the + `tls_error()` buffer USUALLY still looks valid after `tls_free()` returns +- The existing regress only exercises happy-path TLS (`regress/net/net_test.c` + stub serves valid responses) +- The error string ends up in an M_ERROR message displayed to the user, so + garbage would be obvious — but only in the rare connect-failure flow + +Codified test coverage prevents the next analogous mistake from sleeping +in production. + +## What's already done (do not redo) + +- The use-after-free in `tlsconn_connect` is fixed: `*errstr` is now + `xstrdup`-allocated and caller-owned. Signature in `tlsconn.h` is + `char **errstr` (not `const char **`); contract documented in the + header comment around line 39. +- Callers in `src/net/net_api.c` (lines ~169 and ~273) `free(err)` after + `net_emit_error` consumes it. +- A static `tlsconn_dup_error(struct tls *ctx)` helper in `tlsconn.c` + centralises the libtls error capture pattern. See API hardening issue + task 4 for completing that encapsulation. + +## Approach + +The existing harness at `regress/net/net_test.c` runs: + +- `H` (this process) acting as parent+UI +- `N` (forked) running `net_loop` — talks to S over libtls +- `S` (forked) a stub libtls server with a self-signed cert and canned response + +To inject failures, `S` (or a sibling stub) needs modes: + +### Mode 1 — Reject at TLS handshake (cert mismatch) + +S serves a cert whose subject CN does not match the host string N tries to +connect to. Or a cert signed by a CA that N hasn't been configured to +trust. Either path should make `tls_handshake` fail; `tlsconn_connect`'s +fail label runs, `*errstr` should be a valid heap string after return. + +### Mode 2 — Drop connection mid-handshake + +S accepts the TCP connection but closes before completing TLS. Surfaces as +a `tls_error` from the libtls state machine. Same expectation on `*errstr`. + +### Mode 3 — Connect-to-nothing + +H configures N to talk to a port that nothing is listening on. `tls_connect` +fails at the TCP layer. Different libtls error message, same lifecycle. + +## Concrete starting points + +- `regress/net/net_test.c` — the existing stub harness. Lines that fork the + stub server, generate the cert, drive H's expectations. Reuse the + cert-generation Makefile target. +- `regress/net/Makefile` — has the openssl/cert step. Either factor it out + or duplicate for a new `regress/tlsconn/` if the failure-injection tests + don't need a real `net_loop`. +- `src/common/tlsconn.c` — fixed code under test. Note the three `fail` + labels at lines ~70, ~74, ~78 and the early literal at line ~66. +- `src/common/tlsconn.h:39-47` — the documented contract that the test + asserts against. +- `src/net/net_api.c:140` (declares `char *err`), `:169-175` (fail path + with the `free(err)`), `:247`, `:273-278` — the two caller patterns. + +## Test outline + +Recommended location: a new `regress/tlsconn/` dir, mirroring +`regress/json/` (which links the C source directly without forking a +worker). Avoid net_test's full N+S+H choreography unless you specifically +want to exercise the M_ERROR path. + +```c +/* Pseudocode -- adapt to the harness idiom */ + +static void +test_tlsconn_connect_fails_with_owned_errstr(void) +{ + struct tls_config *cfg = tlsconn_client_config(NULL); /* default */ + struct tlsconn c; + char *err = NULL; + int rc; + + /* Mode 3 is the cheapest -- no stub server, no cert. */ + rc = tlsconn_connect(&c, cfg, "127.0.0.1", "1", /* unbound port */ &err); + + CHECK(rc == -1, "connect to unbound port fails"); + CHECK(err != NULL, "errstr populated on failure"); + CHECK(strlen(err) > 0, "errstr is non-empty"); + /* The key assertion: poison the heap, then read err. On the buggy + * code this would have been pointing into freed libtls memory and + * the bytes would no longer match. On the fixed code we own the + * memory so it's stable. */ + explicit_bzero_then_resurrect_test_heap(); + CHECK(strlen(err) > 0, "errstr still valid after heap churn"); + + free(err); + tls_config_free(cfg); +} +``` + +For Modes 1 and 2 add a small stub server inline (the cert-generation in +`regress/net/Makefile` is reusable verbatim — split it into a regress-wide +make include). + +## Constraints + +- **OpenBSD KNF** style. +- **No real network.** Use loopback + ports the kernel picks (bind to + port 0, query the OS for the assigned port — the same pattern + net_test.c uses). +- **Sanitizer-friendly.** The test must pass under `FUGU_DEBUG=1` (UBSan + trap). If you trigger a TLS error path that has a known UBSan signal + in libtls itself, document it and skip rather than mask. +- **No new tlsconn API.** Test against the existing contract. If a new + helper is genuinely needed, file a follow-up for API hardening task 4. + +## Verification + +- `make -C regress/tlsconn regress` — the new test runs and passes +- Revert the fix on a scratch branch (restore `*errstr = tls_error(c->ctx)` + in `tlsconn.c`), rebuild, re-run — the test should fail. Document the + observed failure mode in the commit message. (Don't ship the revert.) +- `make -C regress regress` — total check count rises by the count this + test adds, all dirs still green +- `make FUGU_DEBUG=1 regress` — the same, under UBSan trap + +## Out of scope + +- Verifying every libtls function fugu uses. Focus on the failure paths + that hit the documented contract (`tlsconn_connect`'s `*errstr`). +- Mocking the entire libtls API. Use real libtls against a stub server; + that's how net_test.c works and it's the right scope. +- Touching `src/net/net_api.c` callers. They've been updated; the test is + unit-level on tlsconn.c. blob - /dev/null blob + 5d698d507c141ae6341126006fb465c715709410 (mode 644) --- /dev/null +++ AGENTS.md @@ -0,0 +1,15 @@ +# AGENTS.md + +## Agent skills + +### Issue tracker + +Local markdown under `.scratch//`. No remote tracker; PRs are not a triage surface. See `docs/agents/issue-tracker.md`. + +### Triage labels + +Five canonical roles, default strings (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`), recorded as a `Status:` line in each issue file. See `docs/agents/triage-labels.md`. + +### Domain docs + +Single-context: one `CONTEXT.md` and one `docs/adr/` at the repo root (created lazily). See `docs/agents/domain.md`. blob - /dev/null blob + 1f1f3da5df55ec73c2be616a13d5fcfe56e42e40 (mode 644) --- /dev/null +++ LICENSE @@ -0,0 +1,23 @@ +Copyright (c) 2026 Isaac + +Permission to use, copy, modify, and distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +---------------------------------------------------------------------- + +src/common/log.c and src/common/log.h are reused from OpenBSD base and +are covered by the same ISC terms above, copyright (c) 2003, 2004 +Henning Brauer . + +fugu is inspired by the behavior of "pi" (the @earendil-works/pi-coding-agent +project, MIT, copyright (c) Mario Zechner). It is a clean-room reimplementation +and shares no source with pi. blob - /dev/null blob + 7ac86ed774c4324a94572ea3a4703c4bdd0aa43b (mode 644) --- /dev/null +++ Makefile @@ -0,0 +1,35 @@ +# fugu — OpenBSD-native AI coding agent + +# Keep VERSION in step with V in the port's Makefile. +VERSION= 0.1 +DISTNAME= fugu-${VERSION} +# Where `make dist` writes the release tarball the port installs from. +DISTDIR?= /usr/ports/distfiles + +SUBDIR= src + +.include + +# Package the working tree into the source distfile the port consumes: a clean +# ${DISTNAME}.tar.gz (top directory ${DISTNAME}/, no build artifacts) written to +# ${DISTDIR}. Afterwards, in the port: `make makesum` to refresh distinfo, then +# `make reinstall`. The model never installs itself; this only produces the +# tarball that ports(7) builds and installs from. +.PHONY: dist +dist: + rm -rf ${DISTNAME} + mkdir ${DISTNAME} + pax -rw LICENSE Makefile README.md docs etc man regress run-fugu src \ + ${DISTNAME} + find ${DISTNAME} \( -name '*.o' -o -name '*.d' \) -exec rm -f {} + + find ${DISTNAME} -type d -name obj -exec rm -rf {} + + find ${DISTNAME}/regress -type f -name '*_test' -exec rm -f {} + + rm -f ${DISTNAME}/src/parent/parse.c ${DISTNAME}/src/parent/parse.h \ + ${DISTNAME}/regress/parse/parse.c ${DISTNAME}/regress/parse/parse.h + rm -f ${DISTNAME}/src/parent/fugu ${DISTNAME}/src/ui/fugu-ui \ + ${DISTNAME}/src/net/fugu-net ${DISTNAME}/src/exec/fugu-exec \ + ${DISTNAME}/src/fetch/fugu-fetch + mkdir -p ${DISTDIR} + tar -czf ${DISTDIR}/${DISTNAME}.tar.gz ${DISTNAME} + rm -rf ${DISTNAME} + @echo "wrote ${DISTDIR}/${DISTNAME}.tar.gz" blob - /dev/null blob + 40d2e5ddc746cab904cec4884e9462119c1e0671 (mode 644) --- /dev/null +++ README.md @@ -0,0 +1,86 @@ +# fugu + +`fugu` is an OpenBSD-native AI coding agent — a small, privilege-separated +coding assistant written in C, in OpenBSD style, sandboxed with `pledge(2)` and +`unveil(2)`. It is a clean-room reimplementation (in spirit, not source) of the +TypeScript agent *pi* by Mario Zechner. + +## Status + +Functional. The five privilege-separated processes, the curses and line-mode +front ends, the streaming Anthropic agent loop, the file/shell/search and +brokered web tools, and the append-only session journal with resume, `/clear` +and `/compact` are all in place, with man pages and an OpenBSD port. The +behavior is exercised by the `regress/` suite (including `make`-driven +`pledge`/`unveil` sandbox checks, verifiable at the syscall level with +`ktrace`/`kdump`). + +## Usage + + fugu # start a session in the current project + fugu -l # list saved sessions + fugu -c # resume the most recent session + fugu -r # resume a session by id + fugu -n # print the effective config (secrets redacted) and exit + +In a session, type a message and press Enter; `/clear` resets the active +context, `/compact` replaces it with a model-written summary, and `/quit` +exits. Configuration and the Anthropic key live in `~/.fugu/fugu.conf` (mode +`0600`); see the `fugu(1)` and `fugu.conf(5)` manuals. + +## Platform + +**OpenBSD only.** fugu depends on `pledge(2)`, `unveil(2)`, `libtls`, `imsg(3)`, +base `curses`, and the BSD `make` build system. It deliberately does **not** +build on Linux or other systems — portability shims would undermine the security +model. + +## Architecture + +Five privilege-separated processes, each a separate binary that the parent +`fork`+`exec`s, communicating over `imsg(3)` socketpairs. The parent relays all +worker-to-worker messages (no direct worker-to-worker channels): + +| binary | role | pledge | +|--------|------|--------| +| `fugu` | parent: coordinator + imsg relay; reads config, delegates secrets | `stdio proc` | +| `fugu-ui` | curses/line TUI; owns the tty and the session journal | `stdio tty` | +| `fugu-net` | libtls HTTPS to the Anthropic API; holds the API key | `stdio inet dns` | +| `fugu-exec` | file tools + `ksh`; no network | `stdio rpath wpath cpath proc exec` | +| `fugu-fetch` | libtls web search/fetch (Kagi); parses untrusted HTML in isolation | `stdio inet dns` | + +`fugu-exec`'s execpromises omit `inet`, so neither it nor the commands it runs +can reach the network (opt in per project with `allow_subprocess_net`); its +`unveil` confines writes to the working tree. `fugu-ui` opens its journal under +`~/.fugu/sessions` while it briefly holds `rpath wpath cpath`, then narrows to +`stdio tty`. + +Each role links only the libraries it needs (`libtls` only in net/fetch, +`curses` only in ui), so the untrusted-input processes never map code they don't +use. fugu runs entirely unprivileged (no root, not a daemon). + +## Build (on OpenBSD) + + make obj # optional + make + doas make install # installs under ${PREFIX:-/usr/local} + +The OpenBSD port lives in the ports tree (not this repo), under +`mystuff/productivity/fugu`; build a package with `make package` there. + +## Layout + + src/common/ shared code: log, util, buf, json, imsgproto, http, sse, conf + src/parent/ fugu -> ${PREFIX}/bin (also installs the man pages) + src/ui/ fugu-ui -> ${PREFIX}/libexec/fugu + src/net/ fugu-net -> ${PREFIX}/libexec/fugu + src/exec/ fugu-exec -> ${PREFIX}/libexec/fugu + src/fetch/ fugu-fetch -> ${PREFIX}/libexec/fugu + man/ fugu.1, fugu.conf.5 + etc/ fugu.conf.sample + regress/ unit, integration, and pledge/unveil sandbox tests + +## License + +ISC — see [LICENSE](LICENSE). Bundles the shared OpenBSD `log.c`/`log.h` (ISC, +© Henning Brauer). Inspired by *pi* (MIT, © Mario Zechner); no shared source. blob - /dev/null blob + c5d32ac7e2d236fda6ec003c3df94fa4f0b540b7 (mode 644) --- /dev/null +++ docs/adr/0001-keep-kagi-as-search-adapter-seam.md @@ -0,0 +1,69 @@ +# 1. Keep kagi as a single-adapter seam in anticipation of multi-provider search + +- Status: Accepted +- Date: 2026-06-25 + +## Context + +fugu has one search adapter today: `src/fetch/kagi.{c,h}`. Its interface is the +pure request-path/response-shape half of the Kagi v1 search API — +`kagi_search_body()` writes the JSON request body, `kagi_format_results()` +renders a success envelope into the text the model sees, and `kagi_error_msg()` +extracts the message from an error envelope. The HTTP transport, the +Authorization header, and the secret token live in `fetch_run.c`; the adapter +never touches a socket or a file. + +This is a one-implementation interface. A deletion test asks: if `kagi.{c,h}` +were inlined into `do_web_search()` in `fetch_run.c`, what would get worse? The +answer is that the JSON-shape logic (envelope discrimination, per-field caps, +malformed-response handling) would concentrate in the same function that +already owns transport, deadlines, the SSRF guard, and the Kagi secret. That +function is already the longest in the fetch worker; folding the adapter in +would deepen it further and entangle two concerns — wire protocol vs. response +semantics — that today have a clean seam between them. + +The provider layer in `src/common/` is the precedent. `anthropic.{c,h}` and +`openai.{c,h}` share `provider.h` and the same shape: a pure request builder +plus a streaming decoder that fires a common `struct llm_cbs`. The net worker +is identical for either provider. That shape was introduced before the second +adapter existed; it paid off when openai landed as a drop-in alongside +anthropic rather than a refactor of the net worker. + +The kagi seam exists for the same reason. We have confirmed there is a future +with other search adapters — Perplexity, Exa, and similar — and the seam is +the insertion point for them. + +## Decision + +Keep `src/fetch/kagi.{c,h}` as a search adapter. Its interface stays the pure +request-path/response-shape pair: build a request body, render a success +envelope, extract an error message. Transport, secrets, and ACL stay in +`fetch_run.c`. + +Future search providers (Perplexity, Exa, etc.) plug in at the same seam — a +sibling adapter module exposing the same shape, selected by configuration in +the fetch worker. When the second adapter lands, the common shape should be +lifted into a `search_provider.h` mirroring `provider.h`, the same way +`provider.h` was introduced alongside the second LLM adapter. + +## Consequences + +- Cost: with only one implementation behind it today, this is a hypothetical + seam — leverage we have paid for but not yet collected. The adapter adds one + module, one header, and a small amount of indirection that a strict YAGNI + reading would call premature. +- Benefit: a second search provider becomes a drop-in adapter rather than a + refactor of `fetch_run.c`. The transport, the SSRF guard, the deadline + machinery, and the secret-handling in `do_web_search()` do not have to be + re-opened. The pattern is the same one that made openai a drop-in alongside + anthropic. +- Locality: response-shape logic stays out of `fetch_run.c`, which keeps the + fetch worker focused on transport, sandboxing, and ACLs. The adapter is pure + (no sockets, no files), so it is trivially testable and trivially auditable + for a privilege-separated worker. +- Guidance for reviewers: do not re-suggest collapsing this seam on the basis + that there is only one implementation behind it. The seam is intentional, + matches the established provider-adapter precedent, and is the documented + insertion point for additional search providers. If a second adapter has + still not landed after a long period, revisit this ADR rather than silently + inlining. blob - /dev/null blob + 457df00d424b7a11e6d3e0e00893ff35298c4583 (mode 644) --- /dev/null +++ docs/adr/0002-keep-html2text-as-common-module.md @@ -0,0 +1,53 @@ +# 2. Keep html2text as a common module, anticipating multi-worker consumption + +Date: 2026-06-25 + +## Status + +Accepted + +## Context + +`html2text` (`src/common/html2text.{c,h}`) is a single-pass, allocation-light HTML stripper: ~343 LOC of carefully tested implementation hidden behind a one-function interface: + +```c +void html2text(const char *html, size_t len, struct buf *out); +``` + +The implementation handles hostile web input in strictly linear time, defers whitespace to collapse runs and trailing space, skips `script`/`style`/comment content, decodes the common character references, breaks at block-level elements, and caps the produced text at `HTML2TEXT_MAXOUT` (256 KiB). + +Today the only consumer is `fugu-fetch` (`src/fetch/fetch_run.c:383`), which calls it when the fetched `Content-Type` is HTML or unknown. `fugu-fetch` and `fugu-net` both run with exactly `stdio inet dns`; `html2text` itself needs no syscalls beyond `stdio`, so its sandbox cost is zero in any worker that already has `stdio`. Tests live in `regress/html2text/` and exercise both exact-output and substring contracts. + +The owner has stated that html2text will probably be used in other processes. A plausible second consumer already lurks: `net_run.c`'s tool description advertises the fetch tool as returning "readable text (HTML converted to plain text)" — the same reduction will be useful anywhere a worker handles an HTML payload it did not request (model-server error pages, provider HTML responses to non-streaming endpoints, future tool outputs that proxy web content). + +Without an ADR, a future reviewer applying YAGNI/locality heuristics is likely to suggest folding the 343 LOC implementation back into `fetch_run.c` on the grounds that it has one caller. This ADR exists to head that off. + +## Decision + +Keep `html2text.{c,h}` in `src/common/` as a shared deep module. The interface stays a single function with a fixed output cap; the implementation stays opaque to callers. + +Specifically: + +- Do not inline `html2text` into `fetch_run.c`, even though fetch is currently its sole consumer. +- Do not split `html2text` across workers; there is one canonical implementation, and the `regress/html2text` suite is the authority on its behavior. +- New consumers in other workers (`net`, `exec`, `ui`, `parent`) link the same translation unit via their `Makefile`'s `SRCS`, exactly as `fetch` does today. +- The seam — the one-function header — is the contract. Callers may not depend on intermediate state, the `emit` struct, or the entity table. + +## Consequences + +**Positive** + +- Future workers that need HTML stripping inherit the safe-stripping invariants (linear time, output cap, no allocation amplification, defended against malformed input) for free, without re-deriving them. +- The shape mirrors `mdrender`, the other common deep module with currently one consumer (`ui`) — establishing a consistent pattern for "small interface, deep implementation, shared across workers." +- `regress/html2text` tests cover all current and future consumers; no per-worker duplication of test fixtures. +- Code review is anchored: this ADR exists specifically so reviewers do not re-propose collapsing `html2text` into `fetch_run.c` on locality grounds. + +**Negative / accepted costs** + +- The shared seam is currently hypothetical. Until a second consumer lands, the module pays a small organizational cost (a header in `common/`, an entry in `regress/`) for benefit not yet realized. +- A change to the `html2text` contract (e.g. raising `HTML2TEXT_MAXOUT` or changing whitespace policy) will eventually need coordination across multiple workers rather than one file. + +**Revisit if** + +- 18 months pass with `fetch` still the only consumer, or +- a second consumer's needs diverge enough (different output format, different cap policy) that a shared interface becomes a forcing function rather than a convenience. blob - /dev/null blob + b548c538d606a65bde9bcd4ca3a458ec7c52cd9b (mode 644) --- /dev/null +++ docs/agents/domain.md @@ -0,0 +1,51 @@ +# Domain Docs + +How the engineering skills should consume this repo's domain documentation when exploring the codebase. + +## Before exploring, read these + +- **`CONTEXT.md`** at the repo root, or +- **`CONTEXT-MAP.md`** at the repo root if it exists — it points at one `CONTEXT.md` per context. Read each one relevant to the topic. +- **`docs/adr/`** — read ADRs that touch the area you're about to work in. In multi-context repos, also check `src//docs/adr/` for context-scoped decisions. + +If any of these files don't exist, **proceed silently**. Don't flag their absence; don't suggest creating them upfront. The `/domain-modeling` skill (reached via `/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily when terms or decisions actually get resolved. + +## File structure + +Single-context repo (most repos): + +``` +/ +├── CONTEXT.md +├── docs/adr/ +│ ├── 0001-event-sourced-orders.md +│ └── 0002-postgres-for-write-model.md +└── src/ +``` + +Multi-context repo (presence of `CONTEXT-MAP.md` at the root): + +``` +/ +├── CONTEXT-MAP.md +├── docs/adr/ ← system-wide decisions +└── src/ + ├── ordering/ + │ ├── CONTEXT.md + │ └── docs/adr/ ← context-specific decisions + └── billing/ + ├── CONTEXT.md + └── docs/adr/ +``` + +## Use the glossary's vocabulary + +When your output names a domain concept (in an issue title, a refactor proposal, a hypothesis, a test name), use the term as defined in `CONTEXT.md`. Don't drift to synonyms the glossary explicitly avoids. + +If the concept you need isn't in the glossary yet, that's a signal — either you're inventing language the project doesn't use (reconsider) or there's a real gap (note it for `/domain-modeling`). + +## Flag ADR conflicts + +If your output contradicts an existing ADR, surface it explicitly rather than silently overriding: + +> _Contradicts ADR-0007 (event-sourced orders) — but worth reopening because…_ blob - /dev/null blob + a2f08fb077b319ddaffe5ca74b93a4930cdc5248 (mode 644) --- /dev/null +++ docs/agents/issue-tracker.md @@ -0,0 +1,19 @@ +# Issue tracker: Local Markdown + +Issues and PRDs for this repo live as markdown files in `.scratch/`. + +## Conventions + +- One feature per directory: `.scratch//` +- The PRD is `.scratch//PRD.md` +- Implementation issues are `.scratch//issues/-.md`, numbered from `01` +- Triage state is recorded as a `Status:` line near the top of each issue file (see `triage-labels.md` for the role strings) +- Comments and conversation history append to the bottom of the file under a `## Comments` heading + +## When a skill says "publish to the issue tracker" + +Create a new file under `.scratch//` (creating the directory if needed). + +## When a skill says "fetch the relevant ticket" + +Read the file at the referenced path. The user will normally pass the path or the issue number directly. blob - /dev/null blob + b716855d485f3865f9dfde2a82141721f065b2e7 (mode 644) --- /dev/null +++ docs/agents/triage-labels.md @@ -0,0 +1,15 @@ +# Triage Labels + +The skills speak in terms of five canonical triage roles. This file maps those roles to the actual label strings used in this repo's issue tracker. + +| Label in mattpocock/skills | Label in our tracker | Meaning | +| -------------------------- | -------------------- | ---------------------------------------- | +| `needs-triage` | `needs-triage` | Maintainer needs to evaluate this issue | +| `needs-info` | `needs-info` | Waiting on reporter for more information | +| `ready-for-agent` | `ready-for-agent` | Fully specified, ready for an AFK agent | +| `ready-for-human` | `ready-for-human` | Requires human implementation | +| `wontfix` | `wontfix` | Will not be actioned | + +When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the corresponding label string from this table. + +Edit the right-hand column to match whatever vocabulary you actually use. blob - /dev/null blob + f46a47bb428394fbd2b3c4f08c7236e68d53647a (mode 644) --- /dev/null +++ docs/base-idiom-audit-findings.md @@ -0,0 +1,576 @@ +# fugu ↔ OpenBSD base: idiom audit findings + +**Date:** 2026-06-21 · **Scope:** read-only audit (no code changed) · **Host:** OpenBSD 7.9-current, amd64 (`server`), base at `/usr/src`. + +This is a complete, function-by-function accounting of where fugu's hand-rolled +utilities overlap something OpenBSD **base** already does well — a "seam." Each of +the **233** functions/groups across **21 source units** is classified exactly once: + +- **ADOPT** — base exposes a *linkable* idiomatic equivalent (libc / libutil / libevent / libtls / a base header) that is a drop-in or near-drop-in. +- **REFERENCE** — base solves the same problem but only *inside a base program* (not linkable, e.g. `ftp(1)`'s HTTP client). Align idioms/edge-cases; don't link. +- **KEEP** — no base equivalent (or only a ports lib, out of charter). One-phrase reason recorded. + +**Method / confidence.** Every ADOPT/REFERENCE claim was produced by a per-file +audit agent and then independently re-checked by an adversarial verifier against +the live `/usr/src` tree, headers, and man pages — 42 agents total. **16 citation +corrections** were applied (all kept the verdict; only the file/line was wrong); +they appear inline as **[verified correction: …]** in the Notes column. No verdict +was overturned and no missed seams were found. The `imsg` API was confirmed to be +the **post-rewrite `imsgbuf_*` / `imsg_get_ibuf`** surface — **fugu is already on +it, no migration debt.** + +--- + +## TL;DR + +| Bucket | Count | Meaning | +|---|---|---| +| **ADOPT — actionable** | **1** | Genuine new library adoption (`scandir`). | +| **ADOPT — already idiomatic** | 16 | Already correctly on the base API; listed to prevent regression. | +| **REFERENCE** | 67 | Align to a base program's idiom; a handful are real work, most confirm-only. | +| **KEEP** | 149 | No base option; justified bespoke code. | + +The headline is honest: fugu is **already strongly base-idiomatic**. Only **one** +clean new library adoption exists; the real value is in a few **REFERENCE/SYNC** +alignments (security-relevant secrecy check, lexer modernization, logging drift). + +--- + +## 1. Adopt these (clear wins) + +### Actionable (currently NOT on base — change recommended) + +- **`parent/parent.c` `resolve_last` / `list_sessions` → `scandir(3)` + comparator.** + Replace the hand-rolled `opendir`/`while(readdir)`/`closedir` trio with one + `scandir()` call using a `select()` filter matching the `.jsonl` suffix. **Keep** + the mtime `qsort`/`sessrow_cmp` — `scandir`'s `alphasort` sorts by *name*, and + fugu orders by *mtime*. Cite `/usr/src/usr.bin/diff/diffdir.c:82`, man `scandir(3)` + (decl `/usr/include/dirent.h:91`). **Effort M.** The single clean new adoption in the tree. + +### Already idiomatic (confirmed on the base API — do not regress) + +These are already calling the right linkable base symbol; recorded for completeness: + +- `conf.c` list lifecycle → `sys/queue.h` TAILQ macros. +- `tlsconn.c` (all 7 fns) → libtls `tls_*` (`-ltls`). +- `net_run.c` `net_apply_config`/`cfg_str` → `strtonum(3)`; `net_state_clear` → `freezero(3)` + `tls_config_free`. +- `exec.c` `main` → `getopt(3)`/`signal(3)`/`fstat(2)`; `tools_file.c` `tool_edit` → `memmem(3)`. +- `ui_curses.c` `cw`→`wcwidth(3)`, `utf8_to_wcs`→`mbrtowc(3)`, `submit`→`wcrtomb(3)`, `do_resize`→`resizeterm(3)`. +- `ui.c`/`journal.c` line input → `getline(3)`; `exec_run.c` sandbox → two-arg `pledge`/`unveil` `execpromises`; `imsgproto.c` → current `imsgbuf_*`. + +--- + +## 2. Reference / Sync (align to base source — the real alignment work) + +**Priority 1 — `conf.c` `conf_check_secrecy` → byte-sync to base `check_file_secrecy()`, and add it to `parse.y` `pushfile`.** *(Security-relevant.)* +fugu stores API keys (`anthropic_key_file`, `kagi_token_file`) yet `pushfile` +performs **no** secrecy check. Base copies `check_file_secrecy()` verbatim into ~30 +daemons (`/usr/src/usr.sbin/httpd/parse.y:1795`, called from `pushfile` at `:1834`; +`bgpd/parse.y:4007`). Two deltas to resolve deliberately: **(a)** base takes an +**open fd + `fstat()`** (closes the TOCTOU window) — fugu re-`stat()`s by path; +adopt the fd/`fstat` shape. **(b)** fugu's policy is *stricter* (exact owner, true +0600) than base (owner-or-root, allows group/world *read*) — **keep fugu's stricter +policy**, but make the code read like the base idiom reviewers recognize. It is a +program-internal static (not in `lib/` — confirmed), so this is REFERENCE (copy the +~18 lines), never ADOPT. **Effort S–M.** + +**Priority 2 — `common/parse.y` lexer → sync to current `usr.sbin/httpd/parse.y`.** +fugu's lexer is forked from an *older* pf/httpd template and has drifted on three +coupled points: +1. No `igetc()` indirection layer (`httpd/parse.y:1540`) — fugu inlines raw getc/unget in `lgetc`. +2. Fixed `pushback_buffer[128]` (returns EOF on overflow) vs httpd's **growable per-file `ungetbuf`** moved into `struct file`, doubled via `reallocarray` (`:1605-1616`) — removes the 128-char ceiling. +3. Old `parsebuf`/`parseindex` macro expansion vs httpd's **`START_EXPAND`/`DONE_EXPAND` sentinels + `expanding` flag** pushed back through `igetc` (`:1656,1679-1684`) — enables nested macros and deletes the separate parsebuf path. + +**Correction to the brief's lead:** there is **no `expand_macros` function** in current +base `parse.y` (verified: 0 hits in any `usr.sbin`/`sbin` parse.y) — the real +mechanism is the sentinel/flag scheme above. This is the highest-effort item +(removing `parsebuf` touches `lgetc`+`lungetc`+`findeol`+`yylex` together) and +couples with Priority 1 (threading the `secret` flag through `pushfile`/`conf_parse_file`). +**Effort M–L;** low correctness risk — modernization plus the secrecy win. + +**Priority 3 — `common/log.c` → re-baseline onto current base `log.c`.** +fugu's file is a verbatim copy of `bgpd/log.c` rev 1.64. Current `ntpd/log.c` rev +1.19 has evolved: a `LOG_TO_STDERR|LOG_TO_SYSLOG` **dest bitmask** (dual-sink instead +of strictly either/or), multi-level `-v` verbosity, and dropped the `"fatal in "` +prefix. Decide whether to adopt the ntpd lineage. The `log_warn`/`log_warnx`/`fatal`/ +`fatalx` leaves map to **linkable** libc `warn(3)`/`warnx(3)`/`err(3)`/`errx(3)` +(in `lib/libc/gen/{warn,warnx,err,errx}.c`) — but **keep the wrappers**: the libc +variants are stderr-only and cannot route to syslog or honor `log_procname`. +**Effort S** (cherry-pick drift) to **M** (full re-baseline). + +**Priority 4 — low-effort idiom alignment / confirm-only (no behavior change):** + +- `util.c` x-allocators → ssh `xmalloc.c`: add the `SIZE_MAX/nmemb < size` guard in `xcalloc` to match canonical; consider `xrecallocarray`/`freezero` wrappers for secret buffers (charter). `xasprintf` is **not** present — add the ssh idiom if formatted-alloc is needed. +- `http.c` → ftp `fetch.c`: bracket IPv6-literal hosts + drop default `:80/:443` from the Host header (`fetch.c:740-772`); trim trailing OWS (SP/HT) on header lines (`:880-882`). Chunked-decode + `strtonum` already match. +- `url.c` → ftp `fetch.c`: keep fugu's **stricter** allowlist/anti-injection (better than base's blocklist); align only IPv6/port edge cases (`fetch.c:497-514`). +- `confload.c` → ssh `groupaccess.c:47` (`ga_init`) for the group-name loop; `getgrouplist(3)` already correct. Decide grow-and-retry vs fixed `NGROUPS_MAX`. +- `imsgproto.c` / `net_run` imsg paths → already current `imsgbuf_*`; mirror `relayd/proc.c:593 proc_dispatch` EPIPE/channel-death handling. +- `tools_file.c` `write_atomic` → acme-client `fileproc.c:59` — **fugu adds `fsync(2)`, keep it**; `write_all` → ssh `atomicio.c:50` / vmd `atomicio.c:56`, align EAGAIN/0-byte handling if the fd ever goes non-blocking. +- `tools_shell.c` `run_capture` → ssh `misc.c:2786 subprocess()` — fugu adds timeout + output cap + SIGKILL (keep); align EINTR. **Do not** adopt `popen` (no stderr/timeout). +- `fetch_run.c` `dial`/`http_fetch` → ftp `fetch.c:599 timed_connect` / `:646` TLS loop — already aligned; keep the per-`ai` `addr_blocked` SSRF guard ftp lacks. +- `ui_curses.c` → mandoc `term_ascii.c:93` (locale), ssh `utf8.c`, less `line.c` — already idiomatic; cross-check the `ascii_rep`/`fold_wcs` table against mandoc's; handle `wcwidth==0` combining chars in `render` as less does. +- `json.c` `json_emit_escaped`/`json_tok_strdup` → `vis(3)`/`unvis(3)` are adjacent but **not** drop-ins (different escape grammar) — keep fugu's JSON-correct tables. + +--- + +## 3. Evaluate (tradeoff calls — you decide; recommendation given) + +**`buf.c` → `ibuf` (libutil) / `open_memstream(3)` — RECOMMEND KEEP.** +Neither is a clean drop-in. `open_memstream` (string role; base user +`radiusd_standard.c:345`) loses live O(1) `.data`/`.len` introspection (updates only +on `fflush`) and fatal-OOM. `ibuf` (wire-byte role; `imsg-buffer.c`) has **no +formatted-append** — yet `buf_printf` is the dominant use across json/http/url — and +is shaped for framing, not NUL-terminated strings. `buf` is ~115 lines, +dependency-free, fatal-on-OOM, with a hard-cap growth guard the base options lack. +Adopting either is **effort L for negative payoff**; align idioms, keep `buf`. + +**`parent.c` `relay_loop` poll(2) → libevent — RECOMMEND KEEP poll() for now.** +Every base privsep daemon uses libevent (`dhcpleased/frontend.c:167-190`), and +libevent **is** base (charter-allowed). But fugu is deliberately **foreground, +per-invocation, non-daemon**, fixed 4-channel, no timers/signals beyond SIGWINCH/ +SIGPIPE — a flat `poll()` loop is simpler and callback-free. The one real gap: +write-side backpressure (`imsgbuf_queuelen` → arm `EV_WRITE`); fugu only `POLLIN`s +and relies on `ip_flush` blocking — addressable in the poll model too. Adopt libevent +only if timers/backpressure/more channels arrive. **This is the single largest +stylistic divergence from base — and a justified one.** + +**`parent.c` `run`/`spawn` → relayd/httpd `proc.c` privsep framework — RECOMMEND REFERENCE, don't link.** +`relayd/proc.c:180 proc_init` / `:70 proc_exec` / `:351 proc_kill` are the canonical +privsep helpers, but pull in libevent + the full `struct privsep`/instance-array +model fugu intentionally avoids. fugu's flat 4-worker spawn (with +`closefrom(IMSG_CHILD_FD+1)` + `/dev/null` redirection) is arguably *tighter* than +`proc_exec`. Mirror the ordering/teardown discipline; keep fugu's implementation. + +--- + +## 4. Keep (justified — no base option) + +- **No base library exists:** `json.c` (no base JSON emitter/parser), `sse.c` (no base SSE/EventSource), `html2text.c` (no base HTML parser — lynx/w3m are ports), `anthropic.c` (Anthropic Messages protocol), `kagi.c` (Kagi API), `convo.c` (Messages-API conversation model), `journal.c` (append-only jsonl). +- **Security cores with no base analog:** `fetch_run.c` `addr_blocked` (SSRF private-range classifier — base has none), `imsgproto.c` reassembly/oldest-first eviction (base imsg has no chunking), `conf.c` merge engine (sshd-Match-style, program-internal), path-traversal guards (`session_exists`, `valid_id`). +- **App glue over fugu's own buf/json/imsg layers:** the bulk of `net_run.c`, `exec_run.c`, `fetch_run.c`, `ui.c` — application control flow with no library to adopt. + +--- + +## 4b. OpenBSD base-library linkage check (follow-up) + +A library-first pass (complementing the file-by-file audit above): what each +binary actually links, whether it is all base, and whether any *unused* base +library would replace hand-rolled code. Verified on `server` via `objdump -p` +(direct `DT_NEEDED`), `ldd`, and `/usr/lib`. + +**Linkage is correct, minimal, and 100% base.** Direct `NEEDED` per binary: + +| Binary | Direct NEEDED (base libs) | +|---|---| +| `fugu` (parent) | libc, libutil | +| `fugu-ui` | libc, libcurses, libutil | +| `fugu-net` | libc, libtls, libutil | +| `fugu-exec` | libc, libutil | +| `fugu-fetch` | libc, libtls, libutil | + +`libutil` (imsg) is shared by all; `libtls` only where TLS is spoken; `libcurses` +only in the UI. `libssl`/`libcrypto` appear in `ldd` but are **transitive** via +`libtls` (not direct `NEEDED`), so the port's `WANTLIB = c curses tls util` is +exactly right. **No `/usr/local` (ports) dependency anywhere** — charter satisfied. + +**No new library to adopt.** Every base library fugu does *not* link was checked +and is either not-a-fit or a feature-add, not a replacement for hand-rolled code: + +| Base lib (present) | Verdict | Reason | +|---|---|---| +| `libedit` (`editline(3)`) | KEEP hand-rolled | **Correction: libedit IS in base** (the audit's `handle_input` row wrongly said otherwise). Still not a fit: `el_wgets` owns a blocking read loop and can't interleave with the curses `nodelay`+imsg `poll(2)` loop that edits input *while* streaming deltas render. Line-mode (`getline`) is non-interactive, so editing buys nothing there either. | +| `libevent` | EVALUATE → KEEP | The poll→libevent design question (see §3); base, deliberately unused for a foreground non-daemon. | +| `libpanel`/`libform`/`libmenu` | KEEP | Curses extensions for overlapping windows/forms/menus; fugu's TUI is one full-screen layout — no fit. | +| `libz` | KEEP (note) | gzip/deflate. fugu sends no `Accept-Encoding` and never receives compressed bodies, so nothing to decode today. Available **if** HTTP gzip is ever wanted (feature, not a replacement). | +| `libm` / `libexpat` / `libreadline` / `libcrypto` (direct) | KEEP | No floating point; JSON not XML (jsmn); libedit is preferred over GPL readline and still no fit; no direct crypto (TLS is libtls's job). | + +Net: linkage needs no change; the only artifact is the libedit fact-fix to the +`handle_input` row below. + +--- + +## 5. Complete per-function accounting + +Legend: **Eff** = adoption effort (S/M/L). `path:line` are real sites confirmed on +`server`. **[verified correction: …]** marks a citation the adversarial verifier +fixed. Every function/group across all 21 units appears exactly once below. + +### `src/common/log.c` + +_Verbatim hand-copy of /usr/src/usr.sbin/bgpd/log.c rev 1.64 (only the $OpenBSD$ CVS tag line removed); the canonical daemon-logging idiom is not a linkable lib, so the whole unit is KEEP/SYNC-to-base, with the warn/fatal leaf group REFERENCE-aligned to libc err(3)/warn(3)._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| log_init | Sets debug/verbose state, calls log_procinit(__progname), openlog() when not in debug, tzset(). | `/usr/src/usr.sbin/bgpd/log.c:34` | **KEEP** | S | Hand-copied base code; sync seam, not a libc adopt. | fugu line 32 == bgpd line 34 (offset 2 from the dropped $OpenBSD$ tag). NOTE DRIFT vs newer ntpd log.c rev 1.19: log_init(int n_dest,int n_verbose,int facility) uses a LOG_TO_STDERR\|LOG_TO_SYSLOG bitmask instead of a boolean debug. fugu is on the older bgpd lineage. | +| log_procinit | Stashes the process name used in fatal() messages. | `/usr/src/usr.sbin/bgpd/log.c:49` | **KEEP** | S | Identical to bgpd; sync seam. | ntpd rev 1.19 makes log_procname a non-static extern; cosmetic only, no functional difference. | +| log_setverbose | Setter for the verbose level (toggles log_debug output). | `/usr/src/usr.sbin/bgpd/log.c:56` | **KEEP** | S | Byte-identical to bgpd; sync seam. | No drift from either bgpd or ntpd. | +| log_getverbose | Getter for the verbose level. | `/usr/src/usr.sbin/bgpd/log.c:62` | **KEEP** | S | Byte-identical to bgpd; sync seam. | No drift from either bgpd or ntpd. | +| logit | Varargs front-end: collects ap and forwards to vlog at a given priority. | `/usr/src/usr.sbin/bgpd/log.c:68` | **KEEP** | S | Byte-identical to bgpd; sync seam. | No drift; pure va_start/vlog/va_end shim. | +| vlog | Core sink: in debug mode vfprintf to stderr (asprintf adds newline, OOM fallback); else vsyslog. Preserves errno. | `/usr/src/usr.sbin/bgpd/log.c:78` | **KEEP** | M | Sync seam; the dual-sink routing logic has no linkable base equivalent. | DRIFT vs ntpd log.c rev 1.19 vlog: ntpd va_copy's ap and can emit to BOTH stderr and syslog (dest bitmask); fugu's bgpd-era version is strictly either/or. Adopt the ntpd variant if dual-sink logging is ever desired. | +| log_warn | errno-appending warning: asprintf "%s: %s"(emsg,strerror(errno)) then vlog(LOG_ERR); OOM fallback; restores errno. | `/usr/src/lib/libc/gen/warn.c` warn(3) | **REFERENCE** | M | warn(3)/vwarnc(3) IS linkable but writes only to stderr via __progname; cannot route to syslog or honor log verbosity. | **[verified correction: warn() is in warn.c, not err.c]** err.c lives under lib/libc (linkable, confirmed err.h decls /usr/include/err.h:55-65). Already mirrors the strerror-append + errno-preservation idiom; keep the wrapper purely for the syslog sink. Cannot drop-in replace. | +| log_warnx | Warning without errno suffix; vlog(LOG_ERR). | `/usr/src/lib/libc/gen/warnx.c` warnx(3) | **REFERENCE** | S | warnx(3) is the linkable libc analog but stderr-only; keep wrapper for the syslog sink. | **[verified correction: warnx() is in warnx.c, not err.c]** Decl confirmed at /usr/include/err.h:63. Align idiom only; the LOG_ERR/syslog routing is the reason to keep the wrapper. | +| log_info | vlog(LOG_INFO) informational message. | `/usr/src/usr.sbin/bgpd/log.c:138` | **KEEP** | S | Byte-identical to bgpd; sync seam. | No libc equivalent at LOG_INFO severity (err/warn only cover error/warning); no drift from canonical. | +| log_debug | vlog(LOG_DEBUG) gated on verbose flag. | `/usr/src/usr.sbin/bgpd/log.c:148` | **KEEP** | S | Sync seam. | Minor DRIFT vs ntpd rev 1.19 which gates on verbose>1 (multi-level verbosity); fugu/bgpd gate on verbose!=0. Only matters if multi-level -v is wanted. | +| vfatalc | Static helper: formats 'fatal in : : ' (or without code) and logit(LOG_CRIT). | `/usr/src/usr.sbin/bgpd/log.c:160` err(3) | **KEEP** | M | Sync seam; the LOG_CRIT/syslog formatter has no linkable base equivalent. | DRIFT vs ntpd rev 1.19: ntpd drops the 'fatal in ' prefix, logging just ': ...'. fugu retains bgpd's 'fatal in ' wording. static, so REFERENCE-only mapping to verrc(3); kept as part of the synced unit. | +| fatal | Varargs entry: vfatalc(errno,...) then exit(1). __dead. | `/usr/src/lib/libc/gen/err.c` err(3) | **REFERENCE** | S | err(3)/verrc(3) is the linkable 'print strerror + exit' primitive (__dead, errno-append) but exits via stderr only and can't hit syslog/log_procname. | **[verified correction: err() lives in err.c (decl err.h:43)]** err.h:43 declares err as __dead, matching fugu's __dead fatal in log.h. Align idiom; keep wrapper for the syslog sink. exit(1) vs err()'s configurable eval is the only semantic gap. | +| fatalx | Varargs entry: vfatalc(0,...) then exit(1) (no errno). __dead. | `/usr/src/lib/libc/gen/errx.c:34` errx(3) | **REFERENCE** | S | errx(3) is the linkable libc analog (print + exit, no strerror) but stderr-only. | **[verified correction: errx() __dead void at errx.c:34]** Decl confirmed at /usr/include/err.h:51 (__dead). Align idiom; keep wrapper for syslog routing. | + +### `src/common/util.c` + +_All 4 functions are the ssh xmalloc.c "fatal-on-NULL" wrapper idiom — REFERENCE (xmalloc.c is program-internal, not a linkable lib); the underlying malloc/calloc/reallocarray/strdup are libc and already called directly inside. Wins are alignment to ssh's edge-case handling, not adoption of a library._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| xmalloc | malloc wrapper: rejects size==0, calls fatal() on NULL, never returns NULL. | `/usr/src/usr.bin/ssh/xmalloc.c:27` malloc(3) | **REFERENCE** | S | Wrapper itself is not linkable from base (xmalloc.c lives in usr.bin/ssh/, not lib/); libc malloc already called inside. Align idiom to ssh: ssh's fatal message reports the byte count ('out of memory (allocating %zu bytes)'). fugu's body matches ssh closely otherwise. | Confirmed xmalloc.c exists only at /usr/src/usr.bin/ssh/xmalloc.c (find under /usr/src/lib returned nothing) => REFERENCE not ADOPT. malloc declared in , man 3 malloc. | +| xcalloc | calloc wrapper: rejects nmemb/size==0, fatal() on NULL. | `/usr/src/usr.bin/ssh/xmalloc.c:40` malloc(3) | **REFERENCE** | S | Not linkable; calloc already called inside. Alignment gap: ssh adds an explicit 'SIZE_MAX / nmemb < size' overflow check (xmalloc.c:47-48) before calloc; fugu omits it. calloc itself already detects overflow, so this is belt-and-suspenders, but mirror it to match canonical idiom. | calloc resolves to MALLOC(3), declared . Consider whether the redundant SIZE_MAX check is worth carrying given calloc already guards overflow. | +| xreallocarray | reallocarray wrapper: rejects nmemb/size==0, fatal() on NULL. | `/usr/src/usr.bin/ssh/xmalloc.c:56` malloc(3) | **REFERENCE** | S | Not linkable; reallocarray (the overflow-safe libc primitive, declared /usr/include/stdlib.h:263) already called inside. Idiom divergence: ssh does NOT reject zero size in xreallocarray (only in xmalloc/xcalloc); fugu adds a zero-size fatalx which is stricter/fine. Note base also offers recallocarray(3)/freezero(3) and ssh ships xrecallocarray (xmalloc.c:68) — adopt that wrapper if fugu ever needs zeroing reallocation for secrets per charter. | reallocarray at /usr/include/stdlib.h:263. recallocarray/freezero present in MALLOC(3) — relevant to 'secrets never leaked' charter; flag as a future addition, not a change to this func. | +| xstrdup | strdup wrapper: fatal() on NULL, never returns NULL. | `/usr/src/usr.bin/ssh/xmalloc.c:80` strdup(3) | **REFERENCE** | S | Not linkable; strdup (declared /usr/include/string.h:106) already called inside. fugu's strdup-based body is cleaner than ssh's strlen+xmalloc+memcpy version and equally correct — no change needed; keep the REFERENCE link only for the fatal-on-NULL contract. | strdup at /usr/include/string.h:106, man 3 strdup. Lead mentioned xasprintf but it is NOT present in this file (ssh provides xasprintf/xvasprintf at xmalloc.c:91-112 over libc vasprintf(3) — adopt the same idiom if/when fugu needs formatted alloc). | + +### `src/common/buf.c` + +_Tiny dependency-free string/byte buffer; base has two real equivalents (open_memstream(3) for the dominant string-building role, ibuf/libutil for wire bytes) but neither is a clean drop-in — open_memstream loses O(1) mid-build .data/.len introspection and fatal-OOM, ibuf has no printf — so the abstraction is a justified KEEP with REFERENCE alignment to open_memstream idioms._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| buf_printf | Formatted append via two-pass vsnprintf (size then write); trailing NUL written but not counted in len. | `/usr/src/usr.sbin/radiusd/radiusd_standard.c:345` open_memstream(3) | **REFERENCE** | M | open_memstream wraps a FILE* so vfprintf does the formatting natively and growth is automatic, but it is program-internal-style usage (a libc FILE), errors are silent (NULL/ferror) vs fugu's fatalx, and *psize only updates on fflush — so adopting it removes fugu's two-pass overflow-checked path but loses fatal-on-OOM. Mirror radiusd's open_memstream/fprintf/fclose idiom for string assembly; do not claim ibuf here — ibuf has no printf. | This is the dominant use of buf across json.c/http.c/url.c. ibuf (imsg.h) offers NO formatted-append primitive, so it is not an equivalent for buf_printf. | +| buf_append | Raw byte append with grow; no-op on len==0. | `/usr/src/lib/libutil/imsg-buffer.c:137` ibuf_add(3) | **REFERENCE** | M | ibuf_add(struct ibuf*, const void*, size_t) in linkable libutil is a near-signature match and ibuf_dynamic(len,max) gives the growable/bounded variant fugu's sse.c maxtotal check wants. But ibuf returns int error codes (fugu treats OOM as fatal via xreallocarray) and is designed for wire framing, not NUL-terminated strings; aligning is worthwhile only if the buffer is reframed as bytes. Cite imsg.h:73 decl, imsg-buffer.c impl. | **[verified correction: ibuf_add def is :137 (verified)]** struct ibuf + full API declared /usr/include/imsg.h:34-115; ibuf_data/ibuf_size give .data/.len equivalents but only after-the-fact, not the live field access fugu uses. | +| buf_reserve | Grow capacity to hold need more bytes; doubles from BUF_MIN(64) with explicit (size_t)-1/2 overflow guard -> fatalx. | `/usr/src/lib/libutil/imsg-buffer.c:76` ibuf_reserve(3) | **KEEP** | S | base ibuf_reserve returns a pointer to reserved space (write-cursor semantics), not fugu's capacity-only ensure; the doubling allocator itself is private to imsg-buffer.c. No linkable drop-in for this exact contract. | The overflow guard at buf.c:47 is the edge case to keep; imsg-buffer.c bounds growth via ibuf max arg instead — worth mirroring the bound idiom if ever reframed. | +| buf_init | Zero-initialize the struct (NULL/0/0). | ibuf_dynamic(3) | **KEEP** | S | base buffers are heap-allocated by ibuf_dynamic returning struct ibuf*; fugu uses caller-owned stack structs with init — no base equivalent for in-place init of a caller struct. | Trivial; only exists because fugu embeds struct buf by value (json writer, sse parser) rather than owning a pointer. | +| buf_putc | Append one byte (cast int->unsigned char). | putc(3) | **KEEP** | S | trivial one-line wrapper over buf_append; if open_memstream were adopted this becomes putc(c, fp). Not independently worth a base dependency. | Heavy use in json.c UTF-8 encoding (buf.c via json.c:336-348). Maps cleanly to putc only if the whole buffer becomes a FILE*. | +| buf_puts | Append a C string via strlen+append. | fputs(3) | **KEEP** | S | trivial wrapper; equivalent to fputs(s, fp) under open_memstream. No standalone linkable equivalent for the struct-buf form. | Folds into the open_memstream REFERENCE decision; not an independent ADOPT. | +| buf_reset | Logically truncate to empty (len=0) keeping capacity for reuse. | `/usr/src/lib/libutil/imsg-buffer.c:76` ibuf_truncate(3) | **KEEP** | S | ibuf_truncate exists (imsg.h:98) but operates on the wire-buffer write-position model, not fugu's capacity-retaining reset of a by-value struct; semantics differ enough that it is not a drop-in. | open_memstream has no cheap reset-keeping-capacity (would need rewind+ftruncate on the stream); this favors keeping fugu buf where reuse matters. | +| buf_free | free data and re-zero the struct. | ibuf_free(3) | **KEEP** | S | ibuf_free frees a heap struct ibuf*; open_memstream requires fclose then free(*pbuf). fugu frees only the data of a caller-owned struct — no equivalent for that ownership shape. | Re-zeroing after free (buf.c:112-114) is a use-after-free hardening idiom worth keeping regardless of any base alignment. | + +### `src/common/json.c` + +_Base ships no linkable JSON parse/emit C library (confirmed: no libjson/libjansson, nothing in /usr/src/lib references jsmn/json_parse), so the emitter and jsmn wrapper are all KEEP; two honest adjacencies exist (vis(3) for escaping, wctomb/citrus for UTF-8) but neither is a clean drop-in for JSON semantics._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| json_writer_init | Initialize a JSON emitter writer over a struct buf (depth/type/count state machine). | — | **KEEP** | S | no base JSON emitter; writer state is fugu-specific over fugu's struct buf | State-machine init; nothing in base to align to. | +| json_pre / json_push / json_pop (emitter separator/nesting core) | Place commas, enforce key-before-value, push/pop object/array containers with depth/type checks. | — | **KEEP** | S | no base JSON serializer; this comma/nesting logic is the heart of a JSON emitter base does not provide | Tightly-cohesive static helpers; aborts via fugu-local fatalx (log.h:41, an err.h-style wrapper). No linkable equivalent. | +| json_object_start / json_object_end / json_array_start / json_array_end | Public container open/close wrappers around json_push/json_pop. | — | **KEEP** | S | no base JSON emitter API | Thin public faces over the static core; grouped as the container API. | +| json_emit_escaped | Write a JSON string literal: escape " \ \b \f \n \r \t, \u%04x for <0x20, UTF-8 passthrough. | vis(3) | **REFERENCE** | M | vis encodes C-style/\xNN, not JSON \uXXXX; escape set and UTF-8 passthrough differ, so it cannot be substituted | vis.h is base and linkable (vis(3) confirmed), but JSON escaping rules diverge from VIS_* encodings; mirror only the idiom (single-pass switch, control-char fallthrough), keep fugu's own JSON-correct table. | +| json_key | Emit an object key (escaped) plus ':', with key-after-key / key-outside-object guards. | — | **KEEP** | S | no base JSON emitter | Reuses json_emit_escaped; same KEEP rationale as the emitter core. | +| json_string / json_string_n | Emit a JSON string value from a C string or explicit length. | — | **KEEP** | S | no base JSON emitter | Wrap json_pre + json_emit_escaped. | +| json_int | Emit a JSON integer via buf_printf("%lld"). | — | **KEEP** | S | no base JSON emitter; plain snprintf-style formatting needs no base lib | Number formatting is trivial; no base symbol to adopt. | +| json_bool / json_null / json_raw | Emit literal true/false/null, or pre-formatted raw JSON, after a json_pre separator. | — | **KEEP** | S | no base JSON emitter | json_raw trusts caller-formatted text; no base analog. | +| json_kv_string / json_kv_int / json_kv_bool | Convenience: emit key + typed value in one call. | — | **KEEP** | S | no base JSON emitter | Pure composition of json_key + value emitters; grouped as the kv convenience set. | +| json_parse | Run jsmn over input, growing the token array on JSMN_ERROR_NOMEM (doubling, INT_MAX/2 overflow guard), via fugu's xreallocarray. | — | **KEEP** | S | no base JSON parser; jsmn.h is the charter-permitted vendored tokenizer | Confirmed nothing under /usr/src/lib references jsmn/json_parse; growth loop is idiomatic; uses fugu-local xreallocarray (util.h:25). | +| json_doc_free | Free token array and clear the json_doc. | — | **KEEP** | S | frees jsmn token array; no base type involved | Trivial owner-side free. | +| json_tok_type / json_tok_size / json_tok_streq | Accessors translating jsmn token fields into fugu's enum, plus a token==string compare against d->src. | — | **KEEP** | S | thin accessors over vendored jsmntok_t; no base equivalent | json_tok_streq uses memcmp(3) (base) internally, but the function itself is a jsmn-specific accessor — grouped as the token accessors. | +| json_skip / json_obj_get | Recursively skip a token subtree; look up a value index by key in a jsmn OBJECT. | — | **KEEP** | S | jsmn flat-token traversal; no base JSON DOM/query exists | Core of the jsmn-index navigation; nothing in base to align to. | +| utf8_encode | Encode a Unicode code point to 1-4 UTF-8 bytes into a struct buf (used when decoding \uXXXX escapes). | `/usr/src/lib/libc/citrus/citrus_utf8.c:258` wctomb(3) | **REFERENCE** | M | wctomb only emits UTF-8 when the process locale's ctype is UTF-8 (MB_CUR_MAX); a pledged privsep daemon may not guarantee that, so a deterministic hand-rolled encoder is safer | _citrus_utf8_ctype_wcrtomb at citrus_utf8.c:258 is the base encoder behind wctomb(3) (declared stdlib.h); mirror its bit layout but keep fugu's locale-independent version. Not a clean ADOPT due to locale dependence. | +| hex4 | Parse exactly 4 hex digits into an unsigned int for \uXXXX decoding. | — | **KEEP** | S | fixed-4-digit in-place hex scan; strtoul/strtonum don't fit (no terminator, would over-read) | Deliberately bespoke; base integer parsers can't bound to 4 chars without a copy. | +| json_tok_strdup | Allocate an unescaped copy of a JSON string token: handle \" \\ \/ \b\f\n\r\t and \uXXXX incl. surrogate pairs, building into a struct buf. | unvis(3) | **REFERENCE** | M | strunvis decodes vis(3)/C-style escapes, not JSON's \uXXXX surrogate-pair rules; substituting it would mis-handle Unicode escapes | unvis(3)/strunvis is base and linkable but speaks a different escape grammar; mirror the streaming-decode idiom only. Calls fugu's utf8_encode/hex4 for the JSON-specific parts. | + +### `src/common/imsgproto.c` + +_fugu uses the CURRENT rewritten imsg API (imsgbuf_*, imsg_get_*, imsg_composev, imsg_forward) idiomatically; the thin plumbing wrappers are REFERENCE (already aligned to base, nothing more to adopt), and the chunking/reassembly layer is correctly KEEP because base imsg has no chunking._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| ip_init | Zero the imsgproto state and initialize the imsgbuf over fd via imsgbuf_init(); returns -1 on failure. | `/usr/src/usr.sbin/bgpd/bgpd.c:278` imsg_init(3) | **REFERENCE** | S | Wraps current imsgbuf_init(&ibuf, fd)==-1 idiom 1:1 (matches bgpd.c:278; symbol at imsg.h:141). Already idiomatic; cannot 'adopt' since the value-add is the surrounding chunking struct. No change needed — confirms fugu is on the NEW API, not the removed imsg_init. | Confirmed fugu uses the rewritten API: imsgbuf_init present at /usr/include/imsg.h:141; old imsg_init is gone from the header. Linkable from libutil (/usr/src/lib/libutil/imsg.c). | +| ip_clear | Release all reassembly slots, then imsgbuf_clear() the channel. | imsg_init(3) | **REFERENCE** | S | imsgbuf_clear at imsg.h:147 is used verbatim; the reasm_release loop around it is fugu-specific teardown with no base analog. Correct as-is. | Half of this function (the reasm loop) is KEEP-shaped; the imsg half is already the current idiom. Net: aligned, no action. | +| ip_fd | Accessor returning the underlying channel fd (ip->ibuf.fd). | imsg_init(3) | **REFERENCE** | S | Reads the public imsgbuf.fd field (declared in struct imsgbuf, imsg.h). Base exposes no fd-getter function, so a one-line accessor over the public field is the idiomatic approach (bgpd/iked do the same inline). Keep. | Not an ADOPT: there is no imsgbuf_fd() symbol in base; the field is public and read directly elsewhere in /usr/src. | +| ip_compose | Frame a logical message into one or more imsg chunks (each prefixed with struct fugu_chdr) and queue them via imsg_composev; zero-length sent as one empty chunk. | `/usr/src/usr.sbin/httpd/proc.c:724` imsg_init(3) | **REFERENCE** | M | Per-chunk send uses imsg_composev(&ibuf, type, id, pid, fd, iov, iovcnt) exactly as httpd/proc.c:724 (signature at imsg.h:165, man-confirmed). The chunking/splitting loop has NO base equivalent (MAX_IMSGSIZE=16384 cap, base imsg never fragments). The imsg call is aligned; the framing is intrinsically fugu's. | imsg_composev confirmed in man imsg_init(3) with matching signature. The fugu_chdr framing over it is the KEEP part; the call idiom itself is already correct. | +| ip_flush | Block until all queued frames are written (wraps imsgbuf_flush). | imsg_init(3) | **REFERENCE** | S | Pure 1:1 pass-through to imsgbuf_flush (imsg.h:146). Idiomatic; trivial wrapper for a uniform ip_* surface. No change. | Wrapper exists only to keep callers off the raw imsgbuf; base symbol is current and correct. | +| ip_write | Single non-blocking write pass (wraps imsgbuf_write). | imsg_init(3) | **REFERENCE** | S | 1:1 pass-through to imsgbuf_write (imsg.h:145). Current API, idiomatic. No change. | Part of the event-loop write path; mirrors how bgpd drives imsgbuf_write. | +| ip_read | Read pending bytes into the buffer (wraps imsgbuf_read); 1 ok / 0 EOF / -1 error. | `/usr/src/usr.sbin/bgpd/bgpd.c:1280` imsg_init(3) | **REFERENCE** | S | 1:1 pass-through to imsgbuf_read (imsg.h:144); fugu's 1/0/-1 contract matches the base return convention used at bgpd.c:1280. Already idiomatic. | Confirms current API; the removed imsg_read/imsg_read_nb is not used. | +| ip_get | Pull the next fully reassembled logical message, draining buffered chunks via imsgbuf_get and folding each through handle_chunk. | imsg_init(3) | **REFERENCE** | M | The drain loop uses imsgbuf_get (imsg.h:149, returns 1/0/-1) and imsg_free (imsg.h:172) per the documented contract. The multi-frame reassembly accumulation it wraps has NO base analog. imsg calls are aligned; reassembly is fugu's. | imsgbuf_get is the current accessor (old imsg_get returns ssize_t and still exists at imsg.h, but fugu correctly uses the newer int-returning imsgbuf_get). Loop structure mirrors base imsgbuf_get drain loops. | +| ip_msg_free | Free the reassembled message buffer and zero the struct fugu_msg. | — | **KEEP** | S | Frees fugu_msg.data (fugu-owned heap), not a base struct imsg; imsg_free is for struct imsg and does not apply here. | Distinct from imsg_free(&imsg) which fugu calls separately in ip_get. No base equivalent for the reassembled-message struct. | +| ip_frame_get | Pull one raw frame for the relay without reassembly (wraps imsgbuf_get); 1/0/-1. | imsg_init(3) | **REFERENCE** | S | 1:1 pass-through to imsgbuf_get (imsg.h:149). Idiomatic raw-frame pull used by the parent relay path; current API. No change. | Same accessor as ip_get's loop but exposed unwrapped for the relay that forwards by id without touching payload. | +| ip_forward | Relay a received frame to another channel unaltered via imsg_forward (the parent's routing primitive, by imsg_get_id). | `/usr/src/sbin/dhcpleased/dhcpleased.c:716` imsg_init(3) | **REFERENCE** | S | 1:1 pass-through to imsg_forward(&dst.ibuf, imsg) exactly as dhcpleased.c:716 and iked/control.c:345 (symbol at imsg.h:162). This is precisely the base relay idiom — fugu is correctly using imsg_forward rather than re-composing. No change. | Strong confirmation fugu mirrors base privsep relay design: forward-by-id without reassembly, the same pattern dhcpleased/iked/bgpd use. The current imsg_forward is the right tool. | +| reasm_find | Linear-scan the IP_MAXASM reassembly slots for the in-progress partial matching (src, msgid). | — | **KEEP** | S | no base option: base imsg has no chunking/reassembly layer, so no lookup primitive exists to adopt. | Intrinsic to fugu's logical-message-over-imsg framing. Fixed 8-slot table; no base data structure applies. | +| reasm_alloc | Acquire a reassembly slot, evicting the oldest (by seq) partial when all IP_MAXASM slots are busy to prevent channel wedging. | — | **KEEP** | M | no base option: oldest-first eviction over a bounded reassembly table is fugu's anti-DoS policy; base imsg never reassembles, so nothing to adopt. | Security-relevant bounded-slot eviction. Charter-aligned (defends against a hostile peer holding partials open). Pure fugu logic. | +| reasm_reserve | Grow a partial's buffer toward (but never past) its advertised total, doubling capacity, using xreallocarray. | — | **KEEP** | S | no base option: the grow-to-delivered-bytes (not advertised-total) anti-DoS policy is fugu-specific; the underlying reallocarray is already base via the project's xreallocarray wrapper. | The only base API in play here is reallocarray(3), already adopted transitively through util.h's xreallocarray. The capping policy itself has no base analog. | +| reasm_release | Free a partial's buffer and zero the slot. | — | **KEEP** | S | no base option: frees fugu's reassembly slot; trivial free()+memset with no base equivalent. | Cleanup helper for the reassembly table. free(3) is the only base call and needs no wrapper. | +| handle_chunk | Fold one received imsg chunk into reassembly state: parse fugu_chdr via imsg_get_buf, validate bounds/ordering, fast-path single-chunk messages, and complete or accumulate; 1/0/-1. | imsg_init(3) | **REFERENCE** | L | Frame-field access uses the current accessors imsg_get_type (imsg.h:161), imsg_get_id (158), imsg_get_len (159), imsg_get_buf (155, man-confirmed return 0/-1) — all idiomatic and correct. The reassembly state machine, bounds-checking, and ordering enforcement around them have NO base equivalent. Accessors aligned; reassembly is KEEP-shaped. | fugu correctly uses imsg_get_buf (the copy-out accessor, returns 0/-1 per man imsg_init(3)) rather than poking imsg.data directly — the recommended current idiom. The validation/reassembly logic is the security core and is intrinsically fugu's. Largest function; bulk of it is KEEP, but the imsg surface is exemplary REFERENCE. | + +### `src/common/http.c` + +_A self-contained, hostile-server-hardened incremental HTTP/1.1 client; no linkable base HTTP parser exists, so the protocol logic stays KEEP, but every idiom should be aligned to ftp(1)'s url_get/save_chunked (REFERENCE), and the one true libc win (strtonum) is already adopted._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| http_build_request | Serialise an HTTP/1.1 request (request-line, Host, Connection: close, caller headers, optional Content-Length, body) into a buf. | `/usr/src/usr.bin/ftp/fetch.c:706` | **REFERENCE** | S | fetch.c builds the exact same idiom — "GET %s HTTP/1.1\r\nConnection: close\r\nHost: %s\r\n" via ftp_printf to a FILE* (fetch.c:706-787). It is program-internal, not linkable. Mirror two edge cases fugu omits: (1) IPv6 literal hosts must be bracketed and the scope/%-suffix stripped before the Host header (fetch.c:740-748); (2) omit a default :80/:443 port from Host since some servers choke (fetch.c:766-772). fugu takes host verbatim, so the caller must pre-format these. | Charter-clean (buf_printf only). Not an ADOPT: no linkable request-builder in base; align idioms only. | +| http_resp_init / http_resp_clear / http_resp_done | Lifecycle for the incremental response parser: zero-init state (status=HTTP_STATUS, content_length=-1, default maxline, init line/ctype bufs), free the bufs, and report completion. | — | **KEEP** | S | Plain struct lifecycle around fugu's own buf type; no base library object models an incremental HTTP response. ftp(1) keeps this state as locals in url_get, not a reusable object. | Grouped: three trivial cohesive accessors/lifecycle helpers over struct http_resp. Nothing to adopt. | +| http_resp_push | Core incremental state machine: feed arbitrary response bytes, split lines (capped at maxline), de-frame body by Content-Length / chunked / read-until-close, and hand decoded body bytes to a callback. | `/usr/src/usr.bin/ftp/fetch.c:1154` | **REFERENCE** | M | Base has NO linkable incremental HTTP body de-framer. ftp(1) solves the same problem but over a buffered FILE* with getline()/fread() (blocking, program-internal): save_chunked (fetch.c:1154-1219) and the Content-Length read path. fugu's push-model parser is actually MORE robust than the reference for its threat model (no FILE* buffering, works on partial network reads, every line maxline-capped). Worth mirroring from save_chunked: it strips the optional chunk extension with header[strcspn(header,"; \t\r\n")]='\0' (fetch.c:1167) and verifies the per-chunk trailing CRLF byte-exactly (fetch.c:1196-1207) — fugu already does both. Note fugu correctly ignores Content-Length when chunked is set, matching fetch.c:993-994. | This is the heart of the unit. KEEP-the-code, REFERENCE-the-idioms. No base linkable equivalent (would need a ports HTTP lib = out of charter). | +| http_resp_eof | Signal connection close: completes an EOF-framed body (HTTP_BODY_EOF -> HTTP_DONE), else treats a mid-message close as truncation (-1). | — | **KEEP** | S | Trivial terminal-state transition specific to fugu's push API; base ftp(1) detects this implicitly via getline()/fread() returning EOF, not as a separable function. | No symbol to adopt. | +| http_line | Dispatch one complete (CRLF-stripped) line to the right handler based on parser state: status, header, end-of-headers, chunk-size, or trailer. | `/usr/src/usr.bin/ftp/fetch.c:868` | **REFERENCE** | S | fetch.c's header-reading for(;;) loop (fetch.c:868-936) is the analogous line dispatcher but inlined into url_get and FILE*-driven; not linkable. fugu's split into a state-switch is cleaner for a push parser. Align the trailing-whitespace strip: fetch.c trims trailing \r\n AND spaces/tabs (fetch.c:880-882), whereas fugu only strips a single trailing \r — consider trimming trailing SP/HT too for robustness against sloppy servers. | Internal dispatcher; REFERENCE for the whitespace-trim edge case only. | +| http_status | Parse the status line: require "HTTP/" prefix, skip the version token and spaces, read exactly 3 status digits into r->status. | `/usr/src/usr.bin/ftp/fetch.c:806` strtonum(3) | **REFERENCE** | S | fetch.c parses status via cp=strchr(buf,' '); cp++; strtonum(cp,200,503,&errstr) (fetch.c:806-812) — simpler, and notably it reuses the libc strtonum win for the numeric conversion. But fetch.c does NOT validate the "HTTP/" prefix and clamps the accepted code range to 200..503; fugu's explicit prefix check + 3-digit scan is stricter and better for a hostile peer. Keep fugu's logic; the REFERENCE is the strtonum idiom (already used elsewhere in fugu). | Hand-rolled digit scan is justified by the prefix-validation requirement; strtonum on the substring is an option but loses the 'exactly 3 digits' guarantee. | +| http_header | Parse one header line: split on ':', trim OWS around the value, and record the three headers fugu cares about (Transfer-Encoding: chunked, Content-Length via strtonum, Content-Type). | `/usr/src/usr.bin/ftp/fetch.c:912` strtonum(3) | **REFERENCE** | S | fugu already mirrors fetch.c almost exactly: strncasecmp on the header name then strtonum(val,0,LLONG_MAX,&errstr) for Content-Length (fetch.c:912-918) and strcasecmp for Transfer-Encoding chunked (fetch.c:975-978). The strtonum usage IS the adopted libc win. Difference: fetch.c skips leading OWS with strspn(cp," \t") but doesn't trim trailing OWS; fugu trims both — keep fugu's. One alignment note: fetch.c compares the whole "chunked" value with strcasecmp (exact), fugu uses a ci_contains substring match which is more lenient and matches multi-value TE lists like "gzip, chunked". | Already well-aligned to base idioms; strtonum already adopted. No new linkable symbol beyond strtonum. | +| http_endhdrs | On the blank line ending headers, pick the body-framing mode: no-body statuses (1xx/204/304), then chunked > Content-Length > read-until-close, per RFC 7230. | `/usr/src/usr.bin/ftp/fetch.c:993` | **REFERENCE** | S | fetch.c encodes the same 'chunked wins over Content-Length' rule (sets filesize=-1 when chunked, fetch.c:993-994) but does NOT special-case 204/304/1xx bodyless responses — fugu's handling here is more correct than the reference. Program-internal regardless. Keep fugu's; cite fetch.c only for the precedence rule. | fugu is stricter than base here; pure REFERENCE for the one shared rule. | +| http_chunksize | Parse a hex chunk-size line with overflow checking, stop at ';'/non-hex (extension), require >=1 digit, route size 0 -> trailer and size>0 -> chunk-data. | `/usr/src/usr.bin/ftp/fetch.c:1169` | **REFERENCE** | M | Direct analogue of save_chunked's chunksize = strtoul(header,&end,16) with the header[strcspn(header,"; \t\r\n")]='\0' extension strip (fetch.c:1167-1172). fetch.c relies on strtoul + errno + an explicit chunksize>INT_MAX cap; fugu hand-rolls the hex parse with an inline (size > (LLONG_MAX-d)/16) overflow guard and a LLONG_MAX ceiling. fugu could ADOPT strtoul(3) to shrink the loop, but only after NUL-terminating the line (it has the line buf, so feasible) — fetch.c's INT_MAX clamp is a useful sanity cap fugu may want to mirror to bound a single chunk. Net: REFERENCE; a strtoul refactor is optional and not a clean drop-in because fugu parses a non-NUL-terminated (ptr,len) slice. | strtoul is linkable libc, but the (ptr,len) interface and the desire for an explicit ceiling make the hand-rolled scan reasonable; treat strtoul as an optional simplification, not a mandated ADOPT. | +| ci_contains | Case-insensitive substring search over a length-bounded, non-NUL-terminated haystack (used to spot "chunked" inside a Transfer-Encoding value list). | `/usr/src/lib/libc/string/strcasestr.c:1` strcasestr(3) | **KEEP** | S | strcasestr(3) is real linkable libc (declared /usr/include/string.h:133, man strcasestr(3)) and does exactly this — BUT it requires a NUL-terminated haystack, while ci_contains operates on a (ptr,len) slice of the un-terminated line buffer. Adopting it would force a copy-and-NUL-terminate of the header value first, which is strictly more work and more allocation than the current strncasecmp sliding-window. So no clean drop-in; keep the helper. (memmem(3) is the case-sensitive length-bounded cousin but offers no case-folding.) | Verified strcasestr lives under lib/libc (linkable, not program-static), so the symbol is legit — it just doesn't fit fugu's non-terminated-buffer contract. This is the honest 'looks adoptable but isn't a drop-in' case. | + +### `src/common/sse.c` + +_Incremental push-parser for the SSE (text/event-stream) wire format the Anthropic Messages API streams over; base has zero SSE support and no streaming line-splitter that fits a cross-chunk push model, so the whole unit is KEEP._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| sse_init / sse_clear | Lifecycle: zero the parser, init the three growable buffers (line/event/data), set the callback and the maxline/maxtotal hostile-stream caps; sse_clear frees buffers and re-zeros. | — | **KEEP** | S | no base SSE/EventSource type to init or tear down; buffer mgmt is delegated to buf.* (its own unit) | Trivial struct setup over fugu's own struct sse_parser + struct buf. Nothing in base to align to. Grouped as the tight init/teardown pair: both just memset + buf_init/buf_free over the same fields. | +| sse_push | Feed arbitrary-sized byte chunks; split on CR/LF/CRLF (swallowing the LF of a CRLF via after_cr state), accumulate partial lines across calls into p->line, enforce per-line maxline cap, and hand complete lines to sse_line. | getline(3)/getdelim(3) and strsep(3) exist but neither fits a stateful cross-chunk push parser | **KEEP** | M | no base option: getline/getdelim are pull-model over FILE* (need a seekable stream, not libtls chunks across a privsep boundary); strsep splits a complete in-memory string, not a partial byte stream with CRLF-straddling state | Confirmed no base SSE: ssh server apropos sse and grep -rln 'event-stream\|server-sent\|EventSource' /usr/src both empty. The CR/LF/CRLF + maxline-bounded incremental line assembly is exactly the part base does NOT provide as a library; it is the reason this file exists. Buffer growth itself goes through buf_putc/buf_reset (buf.c unit) where any open_memstream(3)/ibuf overlap would be evaluated, not here. | +| sse_line | Parse one complete line: blank line dispatches the event; ':' comment ignored; split field name at first colon with the one-optional-leading-space rule; handle event:/data: (data lines joined by '\n'), enforce maxtotal cap; accept-and-ignore id/retry/unknown. | — | **KEEP** | M | no base option: this is the EventSource spec field grammar (W3C), not implemented anywhere in OpenBSD base | Field-name/value split could nominally use memchr(3) for the colon scan instead of the hand-rolled for-loop, but that is a micro-cleanup inside buf.* territory, not an idiomatic base equivalent for the function. The spec semantics (leading-space strip, data '\n' join, maxtotal guard) have no base counterpart to adopt or reference. | +| sse_dispatch | Finalize an accumulated event: strip the trailing '\n' from data per spec, default event type to "message", NUL-terminate event and data buffers (uncounted), invoke the user callback, then reset event/data state for the next event. | — | **KEEP** | S | no base option: EventSource dispatch/default-event-type semantics are not in base | Pure SSE-spec dispatch logic plus a callback invocation; nothing linkable in base mirrors it. The NUL-terminate-uncounted idiom is a buf.* convention (documented in buf.h). | + +### `src/common/conf.c` + +_Config model is app-specific KEEP throughout; the one base-overlap is conf_check_secrecy, which mirrors base's verbatim-copied check_file_secrecy (program-internal, REFERENCE) but diverges in signature (path/stat vs fd/fstat -> TOCTOU) and policy._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| conf_check_secrecy | StrictModes guard: a secrets/key file must be owned by uid and mode 0600 before fugu reads it; warns and returns -1 otherwise. | `/usr/src/usr.sbin/smtpd/parse.y:check_file_secrecy (canonical full version; identical body in httpd/parse.y, iked/parse.y; bgpd/parse.y:4007 is the stripped 7.9-current variant)` | **REFERENCE** | S | Base check_file_secrecy is a per-program static in parse.y (not in lib/ or /usr/include), so it is not linkable; align idioms only. Deltas to mirror: (1) base takes an OPEN fd and uses fstat() to close the TOCTOU window -- fugu takes a path and re-stat()s by name; if fugu later open()s the same path it can be swapped. Prefer open()+fstat(fileno). (2) base permits owner==root OR getuid(); fugu requires st_uid==uid exactly -- stricter and arguably correct for a privsep secret, but note the divergence for reviewers. (3) base rejects S_IWGRP\|S_IXGRP\|S_IRWXO (allows group/world READ); fugu rejects S_IRWXG\|S_IRWXO i.e. demands true 0600 -- again stricter, appropriate for secrets. fugu's stricter policy is defensible; the actionable win is matching the fd/fstat shape so the code reads like the base idiom reviewers know. | Confirmed no linkable libutil/libc secrecy helper: grep of /usr/src/lib and /usr/include for check_file_secrecy/secrecy/StrictModes returned nothing. So this can never become ADOPT -- REFERENCE is the ceiling. fugu's redacting/warning via log_warn/log_warnx already matches base style. | +| conf_init / conf_match_new / conf_clear (TAILQ lifecycle group) | Initialize the conf (zero global, TAILQ_INIT matches), allocate+append a match block, and tear down all match blocks + global freeing strings. | `/usr/include/sys/queue.h:439 (TAILQ_FOREACH), :465 (TAILQ_INIT); INSERT_TAIL/REMOVE/FIRST in same header` queue(3) | **ADOPT** | S | Already adopted -- these helpers use base sys/queue.h TAILQ macros idiomatically (TAILQ_INIT, INSERT_TAIL, FOREACH, FIRST, REMOVE). No change needed; recorded as ADOPT because the list machinery IS the base equivalent and fugu uses it correctly. xcalloc/xstrdup wrappers are fugu/ssh-style (ssh xmalloc.c is program-internal, not a base lib), out of scope here. | 83 TAILQ references in queue.h confirm base provenance. Grouped because all three are thin lifecycle wrappers around the same TAILQ head and share no other logic. | +| setstr / conf_set_string / conf_set_flag / conf_settings_clear | Field setters: free-then-xstrdup a string field, OR a CS_* presence bit into the mask, set a scalar's bit, and free all string fields + zero the struct. | — | **KEEP** | S | App-specific bitmask/presence model over fugu's own conf_settings struct; no base API expresses 'set bit N and dup string'. free()/strdup primitives are base but the orchestration is bespoke. | conf_settings_clear must stay in sync with the struct's string members in conf.h:51-62 (model, search_provider, anthropic_key, kagi_token, *_file) -- maintenance hazard if a field is added, but not a base-equivalence question. | +| settings_merge / match_applies / conf_resolve / conf_overlay (merge engine group) | Layered config resolution: copy masked fields from src into dst, test whether a match block applies to a user/group set, collapse global+matches with first-match-wins per key (locked mask), then overlay the user's personal settings. | — | **KEEP** | M | No base option -- the bitmask merge / first-match-wins-per-key semantics are fugu-specific policy over fugu's own structs; sshd's Match resolution (servconf.c) is program-internal and not a linkable analogue worth REFERENCE since fugu's model differs (per-key locking vs sshd's per-line). | Grouped as one cohesive engine: settings_merge is the primitive, match_applies the predicate, conf_resolve/conf_overlay the public drivers. strcmp on group names is plain libc. If anything, conf.h's comment already correctly frames it as 'sshd Match-style' -- documentation alignment only, not adoption. | +| conf_settings_print | Render effective settings for `fugu -n`, printing each set field and redacting secret values as "***" while still showing *_file paths. | — | **KEEP** | S | App-specific serializer over fugu's struct; plain fprintf(3). Secret redaction policy (print *** for key/token, show file paths) is a fugu charter requirement with no base equivalent. | Good charter compliance: secrets never emitted verbatim (conf.c:222-226), consistent with 'secrets never in journal'. fprintf is base libc but the field-by-field logic is bespoke. | + +### `src/common/confload.c` + +_Single function conf_load: config-load orchestration whose libc primitives (getpwuid/getgrouplist/getgrgid) are already idiomatic base calls, but whose group-name resolution and overall load flow should be aligned to OpenSSH groupaccess.c / auth.c idioms — REFERENCE, not a single linkable lib._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| conf_load (group resolution: getgrouplist + getgrgid + xstrdup loop, lines 62-79) | Resolve the invoking user's supplementary groups to group *names* so the config 'Match group' scoping can compare against them. | `/usr/src/usr.bin/ssh/groupaccess.c:47` getgrouplist(3) | **REFERENCE** | S | getgrouplist(3) itself is confirmed linkable libc (/usr/include/unistd.h:499; impl /usr/src/lib/libc/gen/getgrouplist.c) and fugu's call is correct/idiomatic. But the surrounding 'getgrouplist -> loop getgrgid -> xstrdup(gr->gr_name)' helper is exactly OpenSSH ga_init (groupaccess.c:47-67); ga_match (groupaccess.c:73) is the matching half. There is no linkable libutil wrapper for the name-resolution loop, so this is REFERENCE: mirror ga_init's structure. Note divergence: OpenSSH uses a fixed NGROUPS_MAX+1 array; fugu does a grow-and-retry on the documented getgrouplist==-1 'list too small' return — fugu's is arguably more correct but deviates from the base idiom; align deliberately one way or the other. | Verified: man 3 getgrouplist (NAME/SYNOPSIS/RETURN confirm int getgrouplist(const char*,gid_t,gid_t*,int*), -1 == too small). Header /usr/include/unistd.h:499. Base callers: /usr/src/usr.bin/ssh/groupaccess.c, /usr/src/usr.sbin/authpf/authpf.c, /usr/src/sbin/mountd/mountd.c. groupaccess.c:63-65 is the byte-for-byte analogue of confload.c:73-78. getgrgid(3) likewise libc. | +| conf_load (overall load flow: getpwuid/getuid, HOME fallback, optional /etc/fugu.conf via access(), StrictModes secrecy + overlay of ~/.fugu/fugu.conf) | Build the effective config for the invoking user: identify user, locate home, parse optional system defaults, then StrictModes-check and overlay the per-user secrets file. | `/usr/src/usr.bin/ssh/auth.c:352` getpwuid(3) | **REFERENCE** | M | The primitives are all already idiomatic base libc (getpwuid(getuid()) is the canonical pattern, seen in id.c, ftp/main.c, cu.c, etc.; HOME-from-getenv-then-pw->pw_dir fallback and the truncation-checked snprintf("%s/%s") path build are the OpenSSH auth.c idiom at auth.c:342/352). No single linkable library bundles 'load+overlay+StrictModes' — that is per-daemon glue. So KEEP-the-glue / REFERENCE-the-idioms: align the home-dir fallback and path construction to auth.c, and (handled in conf.c's conf_check_secrecy) mirror OpenSSH's secure-path/StrictModes ownership+mode check. resource handling (xreallocarray/xstrdup/free of gids+gnames, conf_clear on every exit) is correct; the rc/goto-done cleanup is sound. | Confirmed idioms: getpwuid(getuid()) base callers include /usr/src/usr.bin/id/id.c, /usr/src/usr.bin/ftp/main.c, /usr/src/usr.bin/cu/cu.c. pw->pw_dir as home fallback: /usr/src/usr.bin/ssh/auth.c:342, auth-rhosts.c:211. snprintf(ret,sizeof(ret),"%s/%s",pw->pw_dir,file) truncation idiom: /usr/src/usr.bin/ssh/auth.c:352. The StrictModes ownership/permission enforcement itself lives in conf_check_secrecy (conf.c, out of scope for this file) and is the OpenSSH secure_filename/auth_secure_path analogue — flag for conf.c's audit. Everything here stays in fugu (KEEP as code) while aligning idioms (REFERENCE). | + +### `src/common/parse.y` + +_Standard OpenBSD parse.y descendant; libc primitives (strtonum/bsearch/vasprintf) already idiomatic, but the lexer is forked from an OLDER httpd template — it has drifted behind current httpd/parse.y on three points: igetc indirection layer, growable ungetbuf, and check_file_secrecy in pushfile. These are REFERENCE alignments (httpd is a program, not a linkable lib), not ADOPT._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| yyerror | Format a parse error with file:lineno prefix and route it to the logger; bumps file->errors. | `/usr/src/lib/libc/stdio/vasprintf.c` vasprintf(3) (under printf(3)) | **REFERENCE** | S | Already uses vasprintf correctly and matches the httpd yyerror shape. Only nit vs template: fugu logs via log_warnx; httpd uses the same pattern. No real change needed; keep aligned to httpd's yyerror. | vasprintf is a true libc symbol (lib/libc/stdio). The yyerror skeleton itself lives only inside base PROGRAMS (httpd/parse.y), so it is a REFERENCE idiom, not a linkable API. fugu's version is faithful. | +| kw_cmp | strcmp comparator for bsearch over the keyword table. | `/usr/src/usr.sbin/httpd/parse.y:1500` bsearch(3) | **KEEP** | S | Trivial 1-line comparator; identical to every base parse.y. Nothing to adopt — it IS the idiom. | bsearch is libc but the comparator is caller-supplied by definition; cannot be 'adopted' from base. Matches httpd/ftpd/relayd verbatim. Leave as-is. | +| lookup | Binary-search the sorted keyword[] table to map an identifier to a token (or STRING). | `/usr/src/usr.sbin/httpd/parse.y:1505` bsearch(3) | **KEEP** | S | Already the canonical sorted-table + bsearch idiom; keyword list is fugu-specific so no base table to reuse. | bsearch confirmed in libc (man 3 bsearch). The 'must stay sorted' table is the standard parse.y pattern; fugu follows it correctly. No drift. | +| lgetc | Lexer input: returns next char honoring quote mode, line-continuation backslash-newline, macro parsebuf, and pushback buffer; pops included files at EOF. | `/usr/src/usr.sbin/httpd/parse.y:1561` | **REFERENCE** | M | DRIFT: fugu's lgetc still embeds the old parsebuf/parseindex + global pushback logic inline. Current httpd factored the raw getc/unget into a separate igetc() (parse.y:1540) and added an eof_reached 'fake EOL' so the last unterminated line counts correctly. Aligning means splitting lgetc into lgetc+igetc and adding eof_reached. | Confirmed: httpd lgetc now calls igetc() and has the eof_reached fake-newline branch (parse.y:1586-1601) that fugu lacks. Pure REFERENCE: lives in a base PROGRAM, not linkable. Low correctness risk but real edge-case (unterminated final line) drift. | +| lungetc | Push one char back onto the input; backstops the macro parsebuf, else uses a fixed 128-byte pushback_buffer. | `/usr/src/usr.sbin/httpd/parse.y:1605` reallocarray(3) | **REFERENCE** | M | DRIFT: fugu uses a fixed global char pushback_buffer[128] (MAXPUSHBACK) that returns EOF on overflow. Current httpd moved the unget buffer INTO struct file as a growable ungetbuf/ungetpos/ungetsize, doubled via reallocarray (parse.y:1610-1616). This is what makes the START_EXPAND/DONE_EXPAND macro scheme possible and removes the 128-char ceiling. | reallocarray is libc (man 3 reallocarray). The growable-ungetbuf design is the httpd template's; align by moving pushback into struct file. This change is the prerequisite for the modern macro-expansion fix below. | +| findeol | Error recovery: consume input up to next newline/EOF and return ERROR so the grammar resyncs. | `/usr/src/usr.sbin/httpd/parse.y:1621` | **REFERENCE** | S | DRIFT (minor): fugu's findeol clears parsebuf and hand-drains the global pushback_buffer. Current httpd findeol is simpler — just loops on lgetc(0) — because pushback now lives in struct file and igetc handles it. Once lungetc/igetc are aligned, fugu's findeol collapses to httpd's 9-line form. | Confirmed httpd findeol body (parse.y:1621-1636) has no parsebuf/pushback poking. REFERENCE only (program-internal). Trivial once the unget refactor lands. | +| yylex | Tokenizer: skips ws/comments, expands $macros, scans quoted strings, numbers (strtonum), and bare identifiers; returns tokens. | `/usr/src/usr.sbin/httpd/parse.y:1656` strtonum(3) | **REFERENCE** | L | DRIFT: fugu expands macros the OLD way (set parsebuf=val; parseindex=0; goto top). Current httpd pushes the macro value back through lungetc bracketed by START_EXPAND(1)/DONE_EXPAND(2) sentinels and guards re-expansion with `expanding` and `!expanding` (parse.y:1656,1679-1684; igetc 1550-1553; defines 1534-1537). This kills the separate parsebuf code path entirely and allows nested/recursive macro use. strtonum usage is already correct and idiomatic. | **[verified correction: NO expand_macros fn exists in httpd parse.y; real mechanism is START_EXPAND/DONE_EXPAND sentinels + `expanding` flag read by igetc (parse.y:1540,1550-1553)]** There is NO function named expand_macros in current httpd/parse.y (grep found it only in vendored perl) — the lead's wording is imprecise; the real mechanism is the inline START_EXPAND/DONE_EXPAND + `expanding` flag interpreted by igetc. strtonum confirmed libc (man 3 strtonum). REFERENCE: template lives in a base program. This is the highest-effort alignment because removing parsebuf touches lgetc, lungetc, findeol, and yylex together. | +| pushfile | Open an include/config file, allocate a struct file, push it on the include stack. | `/usr/src/usr.sbin/httpd/parse.y:1815` fstat(2) | **REFERENCE** | M | DRIFT + CHARTER-relevant: fugu's pushfile has NO secrecy check and links files via a bare nfile->prev pointer (and oddly never inserts onto a list — the caller assigns file=nfile). Current httpd takes a `secret` arg and, when set, runs check_file_secrecy(fileno, name) (parse.y:1833-1839) rejecting group/world-readable or non-owner files, and tracks files on a TAILQ. For fugu, whose charter says secrets never leak, the anthropic_key_file / kagi_token_file settings argue for adopting check_file_secrecy on the config (and key) files. | check_file_secrecy is a per-program helper (httpd/parse.y:1795), NOT a libc symbol — REFERENCE, copy the ~18 lines. Underlying fstat(2) and the S_IWGRP/S_IXGRP/S_IRWXO mode test are base. High value given fugu stores API keys; this is the most security-relevant drift in the file. | +| popfile | Close current file, propagate its error count to the parent, pop the include stack. | `/usr/src/usr.sbin/httpd/parse.y:1855` | **REFERENCE** | S | DRIFT (minor): fugu walks file->prev; httpd uses TAILQ_PREV over a `files` TAILQ and frees the per-file ungetbuf. If the lungetc/ungetbuf and TAILQ alignments are adopted, popfile must also free ungetbuf and TAILQ_REMOVE. Otherwise functionally equivalent. | **[verified correction: popfile is :1855; :1844 is inside pushfile]** REFERENCE: program-internal. Couples to the pushfile/lungetc refactors; on its own fugu's prev-pointer version is correct. | +| symset | Define/replace a macro symbol (name=val), honoring persistent (-D cmdline) entries. | `/usr/src/usr.sbin/httpd/parse.y` | **KEEP** | S | Matches the base template; uses fugu's xstrdup/xcalloc wrappers. No base library API for a symbol table — it is the standard per-parser TAILQ idiom. | Identical in spirit to httpd symset. No linkable base equivalent (a macro symbol table is parser-local). fugu also lacks the cmdopts/symset_cmdline path but that is feature scope, not drift. Keep. | +| symget | Look up a macro symbol's value by name. | `/usr/src/usr.sbin/httpd/parse.y` | **KEEP** | S | Standard TAILQ_FOREACH lookup; no base library symbol-table API to adopt. | Matches base template verbatim. Keep. | +| conf_parse_file | Public entry point: set conf/cur, pushfile(filename), run yyparse, collect errors, drain the macro table so a re-parse starts clean. | `/usr/src/usr.sbin/httpd/parse.y:1880` | **REFERENCE** | S | fugu's driver matches the base parse_config shape (pushfile/topfile/yyparse/popfile/return errors?-1:0). Two alignments: (1) once pushfile gains a `secret` arg, pass the right flag here; (2) httpd's parse_config additionally walks symset cmdline opts and warns on unused symbols — optional. The macro-drain-on-exit is fugu's own (good hygiene for repeated parses). | REFERENCE: every base parser has this driver but it is program-internal, not linkable. fugu's is correct and idiomatic; the only meaningful follow-on is threading the check_file_secrecy `secret` flag through. | + +### `src/common/url.c` + +_Both functions are sound, hardened reimplementations with no linkable base equivalent; base only solves the same problems inside programs (ftp/httpd) — REFERENCE for idioms, but fugu's allowlist/anti-injection posture is stricter than base and worth keeping._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| url_parse | Parse an absolute http(s) URL into {https, host, port, path} with bounded fixed buffers; default port from scheme; drop fragment; reject userinfo-smuggling, non-host chars in host, and space/control bytes in path (anti request/Host smuggling). | `/usr/src/usr.bin/ftp/fetch.c:306` strtonum(3), strlcpy(3) | **REFERENCE** | M | No base library parses URLs (the only man -k url hits are curl, a ports lib). url_get() at fetch.c:306 is static/program-internal: scheme split fetch.c:351-373, userinfo+path split fetch.c:377-405, IPv6-bracket + port split fetch.c:497-514. Mirror its [IPv6]/port handling, but note divergences: fetch.c uses strrchr for ']' and last ':' (portnum, fetch.c:508-509) and defers host validity to getaddrinfo; fugu bounds the authority with strcspn/memchr and additionally enforces a host-char allowlist (url.c:101-106) and rejects control/space in path (url.c:108-113) — hardening base does NOT do. Keep fugu's stricter posture; align only edge-case handling. Underlying primitives strtonum/strlcpy/strlcat/strcspn/memchr are all base and already used idiomatically. | strtonum(3) confirmed in ; strlcpy/strlcat(3) in ; strcspn declared at /usr/include/string.h:78. Port range check via strtonum(u->port,1,65535,&errstr) is the canonical base idiom. fugu's leftmost-bounded parse is arguably safer than fetch.c's strrchr against crafted authorities. Keep the function; treat fetch.c as a robustness checklist, not a source to link. | +| url_encode | RFC 3986 percent-encode: emit alnum and unreserved (-_.~) verbatim, %XX-encode everything else (uppercase hex), writing into a struct buf. | `/usr/src/usr.sbin/httpd/httpd.c:682` | **REFERENCE** | S | No linkable percent-encoder in libc/libutil. Two base analogs, both program-internal: httpd.c:682 url_encode (same static char hex[]="0123456789ABCDEF" table, but a BLOCKLIST — only encodes space #%?\"&< plus <=0x1f/>=0x7f — and calloc's a buffer) and fetch.c:141 url_encode (to_encode() predicate, malloc, lowercase %02x). fugu's ALLOWLIST (keep only RFC3986 unreserved) is stricter and more correct for query/form-data than httpd's blocklist, and it streams into buf instead of allocating. Adopt nothing; cite httpd.c:682 as the table/idiom reference and keep fugu's allowlist. | isalnum from (already included, url.c:19) is base/standard. fugu's uppercase-hex + unreserved-keep matches RFC 3986 sec 2.3 exactly; httpd's blocklist would leave e.g. '+' and ';' unencoded, which is wrong for form bodies. Keeping fugu's implementation is the right call; the base programs validate the hex-table idiom but should not be adopted. | + +### `src/common/html2text.c` + +_A deliberately-minimal HTML-to-text reducer; OpenBSD base has no HTML parser or entity decoder (lynx/w3m are ports), so the tag/entity/whitespace machinery is all KEEP; only the UTF-8 codepoint encoder has a base analog (wctomb(3) / less), and only the output sink overlaps open_memstream(3)._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| html2text | Main single-pass linear scanner: strips tags, skips comments/declarations/PIs, scans tag names honoring quoted attrs, dispatches script/style raw-skip, break/paragraph tags, entities, and whitespace collapse. | — | **KEEP** | L | no base HTML parser; lynx/w3m are ports (/usr/ports/www/lynx, /usr/ports/www/w3m), nothing in /usr/src/usr.bin, no html header in /usr/include | Charter-correct: hand-rolled, strictly linear over hostile input, output capped at HTML2TEXT_MAXOUT. Base offers zero linkable HTML tokenizer, so this stays bespoke. | +| emit_codepoint | Encodes a Unicode codepoint to UTF-8 by hand (rejecting controls, surrogates, out-of-range), 1-4 byte forms. | `/usr/src/usr.bin/less/cvt.c:97` wctomb(3) | **REFERENCE** | M | wctomb(3) is locale-dependent (LC_CTYPE) and only emits UTF-8 under a UTF-8 locale; the hand encoder is locale-agnostic and pledge-safe, so adopting wctomb would be a regression for this hostile-input path. Mirror less/cvt.c only for the encode idiom, keep the manual path. | wctomb declared in (confirmed man 3 wctomb). Canonical base call site is less/cvt.c:97 `dst += wctomb(dst, ch)`. Keep fugu's validation of surrogates/range which wctomb does not guarantee across locales. | +| emit_named | Resolves a named HTML character reference (amp/lt/gt/nbsp/copy/mdash/...) from a small static table to its replacement text, else echoes &name; literally. | — | **KEEP** | M | no base HTML-entity decoder; mandoc chars.c maps roff special-char escapes (\(co), not HTML entity names | Table-driven and intentionally tiny. strncasecmp inside it is base libc (strings.h:75) but the entity table and fallback logic are fugu-specific. | +| decode_entity | Parses an &...; reference starting at '&': numeric (&#...; / &#x...;) with digit/overflow clamping, or alphanumeric named refs, dispatching to emit_codepoint/emit_named; falls back to literal '&'. | — | **KEEP** | M | no base charref parser | Uses isxdigit/isdigit/tolower/isalnum (ctype.h:68-79, base libc) but the &#/&#x/&name; grammar and clamping are HTML-specific glue with no base equivalent. | +| skip_raw | Skips raw CDATA-like content (script/style body) to the matching delimited close tag, refusing bare-prefix false matches like so hidden text cannot leak. | — | **KEEP** | M | no base HTML tokenizer; raw-text-element handling is HTML-spec-specific | Security-relevant (prevents script text leakage). strncasecmp/isspace are base libc; the close-tag delimiter logic is bespoke and correct to keep. | +| is_break_tag / is_paragraph_tag / tag_is | Tight group: tag_is does a length+case-insensitive name compare; is_break_tag and is_paragraph_tag test membership in static block-level tag sets to decide line vs paragraph breaks. | strncasecmp(3) | **KEEP** | S | the case-insensitive compare is already base (strncasecmp, strings.h:75); the tag-name sets and break/paragraph classification are HTML semantics with no base source | tag_is already wraps base strncasecmp idiomatically (declared /usr/include/strings.h:75). The membership tables (p/br/div/li/... vs paragraph tags) are pure HTML policy; keep as-is. | +| emit_byte / emit_space / emit_break | Tight group: deferred-whitespace state machine. emit_space/emit_break record pending whitespace intent; emit_byte flushes the strongest pending whitespace (break beats space, capped at 2 newlines) then appends one byte to the output buf, honoring HTML2TEXT_MAXOUT. | `/usr/src/lib/libc/stdio/open_memstream.c:1` open_memstream(3) | **KEEP** | M | output accumulation overlaps open_memstream(3)/fugu's own struct buf, but the whitespace-deferral state machine is the actual value here and has no base equivalent | Lead's noted overlap: open_memstream(3) (stdio.h:256, lib/libc/stdio/open_memstream.c) is the base growable-output sink, but fugu already uses its own struct buf (xreallocarray-backed, OOM-fatal) with a hard MAXOUT cap that open_memstream does not give. The space/break collapse logic is bespoke and worth keeping; do not swap the sink for open_memstream (loses the cap and the fatal-OOM guarantee). | + +### `src/common/tlsconn.c` + +_Thin blocking libtls client shim; every function is a near-1:1 wrapper over base libtls (linkable, -ltls) used idiomatically per the man pages and ftp(1) fetch.c. All ADOPT (already on base); minor alignment nits only._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| tlsconn_client_config | Build a verifying client tls_config; optionally load a CA bundle file into memory via tls_load_file + tls_config_set_ca_mem so the connect path needs no rpath (pledge "stdio inet dns"). | `/usr/src/lib/libtls/tls_util.c:151` tls_load_file(3) | **ADOPT** | S | Already correct base usage. One alignment nit: the man page pairs tls_load_file() with tls_unload_file() (zeroes the buffer) rather than plain free(ca) at lines 46/50. CA data is not secret so impact is cosmetic, but tls_unload_file() is the documented idiomatic counterpart and is a linkable base symbol (declared in /usr/include/tls.h, impl tls_util.c:223). Optional: also consider tls_config_set_protocols() if you want to pin TLSv1.2+1.3 explicitly rather than relying on the libtls default. | **[verified correction: tls_load_file def is :151 (tls_config_* live in tls_config.c)]** tls_load_file/tls_config_set_ca_mem/tls_config_new all confirmed in base man page (man -k) and declared in /usr/include/tls.h (grep -c tls_load_file = 1). Comment at line 44 correctly notes set_ca_mem copies the data, justifying the free of the caller's copy. | +| tlsconn_connect | Create a tls_client, configure it, tls_connect(host,port) (libtls does its own socket+DNS), then drive the handshake to completion; libtls owns and closes the fd. | `man tls_connect(3)` | **ADOPT** | S | Already correct base usage; the tls_configure->tls_connect*->handshake-loop->tls_error sequence mirrors fetch.c:650-665 exactly. No change needed. | **[verified correction: ftp/fetch.c never calls tls_connect() (uses tls_connect_socket); cite the man page]** tls_connect confirmed in man -k output (tls_connect(3) groups tls_connect_socket etc.). Error reporting via tls_error(ctx) matches fetch.c idiom. Fail path frees ctx and nulls it correctly. | +| tlsconn_connect_socket | Same as tlsconn_connect but over a caller-supplied connected fd (used when the worker does its own connect under tighter pledge); records fd ownership in c->fd, clears it on failure since caller still owns the passed fd. | `/usr/src/usr.bin/ftp/fetch.c:655` tls_connect_socket(3) | **ADOPT** | S | Already correct base usage; near-identical to fetch.c:655 (tls_connect_socket(tls, fd, sslhost) then the tls_handshake WANT_POLL loop). fd-ownership handling on the fail path (c->fd = -1 so caller keeps the fd) is a correct extension over fetch.c. No change needed. | tls_connect_socket confirmed in man -k and tls_connect_socket(3) man page exists. fetch.c uses servername (sslhost) the same way fugu passes servername. | +| tlsconn_read | Blocking read wrapper: call tls_read and retry immediately on TLS_WANT_POLLIN/POLLOUT so callers see ordinary blocking-read semantics. | `/usr/src/usr.bin/ftp/fetch.c:1779` tls_read(3) | **ADOPT** | S | Already correct base usage; the do{}while(WANT_POLLIN\|\|WANT_POLLOUT) loop is the exact documented blocking-fd pattern (tls_read(3) RETURN VALUES: "the same function call should be repeated immediately") and is byte-for-byte fetch.c's stdio_tls_read_wrapper (fetch.c:1778-1782). No change needed. | **[verified correction: off-by-one: tls_read call is :1779]** Confirmed against tls_read(3) man page text and fetch.c:1767/1780. Returns -1 on error / size on success straight through, matching man page. | +| tlsconn_write | Blocking write wrapper: call tls_write and retry immediately on TLS_WANT_POLLIN/POLLOUT. | `/usr/src/usr.bin/ftp/fetch.c:1766` tls_write(3) | **ADOPT** | S | Already correct base usage; matches the tls_write(3) man-page blocking example (continue on WANT_POLL*) and fetch.c's stdio_tls_write_wrapper (fetch.c:1759-1769) line-for-line. No change needed. | **[verified correction: off-by-one: tls_write call is :1766]** Confirmed against tls_write(3) EXAMPLES section and fetch.c:1767. | +| tlsconn_write_all | Loop tlsconn_write over a buffer until all len bytes are written, since tls_write may return a short count. | — | **KEEP** | S | No base TLS write-all primitive; this is a trivial short-write drain loop over tls_write. The pattern (while len>0: ret=tls_write; buf+=ret; len-=ret) is exactly the tls_write(3) man-page blocking example, so it is idiomatic, just not a library symbol. | Correct: tls_write can short-write so the loop is necessary. Minor robustness note (not base-related): on n==0 it would spin, but tls_write does not return 0 per the man page (size on success, -1 on error, or WANT_POLL* handled inside tlsconn_write), so this cannot loop forever in practice. | +| tlsconn_close | Tear down: tls_close (TLS-layer shutdown) then tls_free the context, and close the owned fd if any. | `/usr/src/lib/libtls/tls.c:921` tls_close(3) | **ADOPT** | S | Already correct base usage. Aligns with tls_close(3): "Only the TLS layer will be shut down and the caller is responsible for closing the file descriptors, unless the connection was established using tls_connect()." fugu correctly close()s c->fd only for the connect_socket case (fd>=0) and leaves the tls_connect() case (fd==-1) for libtls to close. Minor: tls_close can return TLS_WANT_POLL* / -1, which fugu ignores; acceptable on teardown but could be drained like the other loops for symmetry. | **[verified correction: tls_close def (path_line was empty)]** tls_close(3) is part of the tls_read(3) man page group (verified). fd ownership logic is consistent with the connect/connect_socket fd bookkeeping. | +| do_handshake | Static helper: drive tls_handshake to completion, retrying on TLS_WANT_POLLIN/POLLOUT (blocking semantics). | `/usr/src/usr.bin/ftp/fetch.c:660` tls_handshake(3) | **ADOPT** | S | Already correct base usage; the do{}while(WANT_POLL*) loop is identical to fetch.c:658-661. Explicit handshake before first read/write is optional per tls_handshake(3) (read/write auto-handshake) but is a clean way to surface handshake errors early. No change needed. | **[verified correction: off-by-one: tls_handshake call is :660]** Confirmed against tls_handshake(3) (in the tls_read(3) group) and fetch.c:658-661. Returns the raw 0/-1 which callers map to errstr via tls_error. | + +### `src/common/anthropic.c` + +_App-specific Anthropic Messages API request serializer + SSE event interpreter; built entirely on fugu's own buf/json layers. No linkable base equivalent exists (no base JSON/SSE library). The one base seam, strtonum in obj_getint, is already idiomatically adopted. Everything else is KEEP._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| anthropic_build_request | Serialise a streaming Messages request body (model, max_tokens, stream, system, messages array, tools) into a struct buf via the json_writer emitter. | — | **KEEP** | S | Anthropic wire-format construction; no base JSON emitter exists (no json header in /usr/include). | Pure glue over fugu's json_writer. The buf-vs-open_memstream question belongs to buf.c/json.c, not here; this function never touches raw stdio. Charter-clean. | +| anthropic_build_request_raw | Same as build_request but splices pre-formatted messages_json verbatim as the messages value (the net worker's conversation projection). | — | **KEEP** | S | Anthropic-specific body shape; no base equivalent. | Near-duplicate of build_request differing only in messages source; could be internally factored but that is a local refactor, not a base seam. | +| anthropic_stream_init / anthropic_stream_clear | Lifecycle for struct anthropic_stream: zero the struct, copy callbacks, init/free the three accumulator bufs (tool_id, tool_name, tool_json), set cur_index=-1. | — | **KEEP** | S | Trivial struct init/teardown over fugu buf; nothing in base to adopt. | memset(0)+buf_init/buf_free idiom. Standard C; no libutil/libevent helper applies. | +| anthropic_stream_event | Top-level SSE-event dispatcher: parse the event's JSON data, validate it is an object, then route by event name (content_block_delta/start/stop, message_start/delta/stop, error; ping/unknown ignored). | — | **KEEP** | M | No base SSE/event-stream parser exists (grep of /usr/src for event-stream/content_block_delta: zero hits). | Strcmp dispatch table over Anthropic's event vocabulary. Entirely protocol-specific; base offers no linkable equivalent and no program-internal one to mirror. | +| ev_message_start | Handle message_start: read message.usage.{input,output}_tokens and fire on_usage callback. | — | **KEEP** | S | Anthropic usage-block extraction; app-specific. | Navigates fugu json_doc via json_obj_get + obj_getint helper. No base seam. | +| ev_block_start | Handle content_block_start: record block index, detect tool_use blocks, capture tool id/name into accumulator bufs. | — | **KEEP** | S | Anthropic content-block state machine; app-specific. | obj_getstr -> buf_puts -> free pattern over fugu json. Correctly frees each strdup. No base equivalent. | +| ev_block_delta | Handle content_block_delta: route text_delta to on_text callback and input_json_delta into the tool_json accumulator (thinking_delta ignored in v1). | — | **KEEP** | S | Anthropic delta-type routing; app-specific. | Streaming fragment dispatch; no base equivalent. json_tok_strdup borrowed from fugu json layer. | +| ev_block_stop | Handle content_block_stop: if the closing block was a tool_use, emit the assembled tool call (id, name, accumulated JSON, defaulting to {} when empty) via on_tool_call, then reset accumulators. | — | **KEEP** | S | Anthropic tool-call finalisation; app-specific. | Uses bufcstr borrow-trick to NUL-terminate accumulators. No base seam. | +| ev_message_delta | Handle message_delta: update output_tokens (fire on_usage) and surface delta.stop_reason via on_stop. | — | **KEEP** | S | Anthropic stop/usage extraction; app-specific. | json_tok_type guard before strdup is correct. No base equivalent. | +| ev_error | Handle error events: extract error.type/message, mark the stream fatally errored, fire on_error with safe fallbacks. | — | **KEEP** | S | Anthropic error-envelope handling; app-specific. | Frees both strdup'd strings; NULL-safe fallbacks. No base seam. | +| obj_getstr | Helper: duplicate an object member (string or primitive) as a free()-able C string, or NULL if the key is absent. | — | **KEEP** | S | Thin wrapper over fugu json_obj_get/json_tok_strdup; no base JSON accessor exists. | Belongs to the json layer's call-convenience, not a base-replaceable unit. | +| obj_getint | Helper: read an object member as long long, parsing the token text with strtonum() and falling back to a default on absence or parse error. | `/usr/src/usr.bin/calendar/io.c` strtonum(3) | **KEEP** | S | Already adopts the correct base API; no further base alignment available. | Verified strtonum is base: declared /usr/include/stdlib.h:305, man strtonum(3), idiomatic across /usr/src/usr.bin (aucat/audioctl/cal/calendar). The function already uses strtonum(LLONG_MIN,LLONG_MAX,&errstr) with the canonical errstr-checked pattern, so this is best-practice already, not a pending ADOPT. Listed KEEP because the only base seam is already taken. | +| bufcstr | Helper: borrow a struct buf's bytes as a NUL-terminated C string by appending '\0' then uncounting it (len--), so the data pointer is a valid cstr without growing len. | — | **KEEP** | S | buf-internal idiom; depends entirely on fugu's struct buf, not a base type. | If buf.c were ever reimplemented on open_memstream(3) (stdio.h, confirmed in base), this borrow-trick would become a simple fflush+pointer read — but that is a buf.c-layer decision, out of scope for this file. Here it stays KEEP. | + +### `src/parent/parent.c` + +_Privsep coordinator is already well-aligned to base idioms (imsgbuf_*, freezero, closefrom). One clean ADOPT (scandir for the 3 hand-rolled dir scans); the spawn/relay core is a strong REFERENCE to relayd/proc.c; the poll loop vs libevent is a deliberate-design REFERENCE, not a must-adopt given fugu's foreground/non-daemon charter._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| usage | Print usage string to stderr and exit(1). | — | **KEEP** | S | trivial per-program boilerplate; no library equivalent | Standard OpenBSD usage() pattern already (extern __progname, fprintf, exit). Nothing to adopt. | +| main | getopt parse, conf_load, dispatch -n/-l/-r/-c, then run() the privsep coordinator. | getopt(3) | **KEEP** | S | already uses base getopt(3) idiomatically | Option handling is textbook base getopt. argc/optind cleanup correct. No change. | +| create_session_dir | Best-effort mkdir of ~/.fugu and ~/.fugu/sessions (0700). | — | **KEEP** | S | plain mkdir(2); no base mkdir -p library helper exists | Base has no libc 'mkpath'/mkdir-p; mkpath(3) is libutil-adjacent but not present here. Two literal mkdir calls are fine and idiomatic. | +| sessions_path | Compose "$HOME/.fugu/sessions" with overflow check. | — | **KEEP** | S | snprintf-with-truncation-check is the base idiom already | Correct (>= size) truncation guard. No win available. | +| session_exists | Validate id has no '/'/'.'/'..' then stat ~/.fugu/sessions/.jsonl as a regular file. | — | **KEEP** | S | path-traversal guard + stat is application policy, not a library seam | The strchr('/')/dot checks are a deliberate security guard; no base helper replaces it. Keep. | +| resolve_last / list_sessions (directory scan group) | Enumerate ~/.fugu/sessions/*.jsonl: resolve_last picks newest by mtime; list_sessions builds rows and prints. Both hand-roll opendir+readdir(+qsort in list). | `/usr/src/usr.bin/diff/diffdir.c:82` scandir(3) | **ADOPT** | M | scandir mallocs the whole namelist up front and you supply a select() filter + comparator; for resolve_last you still walk+stat each entry so the win is mostly the readdir/closedir boilerplate, not the mtime logic. The qsort in list_sessions stays (mtime sort, not name sort, so alphasort doesn't replace sessrow_cmp). | scandir confirmed declared /usr/include/dirent.h:91 and man scandir(3); real base caller /usr/src/usr.bin/diff/diffdir.c:82 (scandir(path1,&dirp1,selectfile,alphasort)). ADOPT collapses the opendir/while(readdir)/closedir trio into one call with a select() that matches the ".jsonl" suffix. Note: the mtime-newest sort is fugu-specific, so keep sessrow_cmp + qsort; scandir replaces the iteration, not the ordering. | +| session_preview | Parse a stored JSON message, extract "content" string, fold control chars to space, collapse whitespace into a one-line preview. | — | **KEEP** | S | JSON + control-char sanitization is app logic over vendored jsmn; no base option | vis(3)/strnvis exist in base for visual encoding but the goal here is space-folding for a fixed-width column, not vis escaping. Keep. | +| scan_session | Count non-empty lines (messages) in a .jsonl file and preview the first via getline. | getline(3) | **KEEP** | S | already uses base getline(3); newline trim is inherent, no one-call variant | getline confirmed (GETDELIM(3)). The manual \n/\r trim loop is the standard idiom; base offers no trimming reader. Keep. | +| sessrow_cmp | qsort comparator ordering sessions newest-mtime-first. | qsort(3) | **KEEP** | S | mtime comparator can't be alphasort(3); it's domain ordering | alphasort sorts by name; this sorts by mtime descending, so it must stay hand-written. Correct (avoids subtraction overflow by branching). Keep. | +| run | Spawn all 4 workers with policy argv, delegate config+secrets, pledge "stdio proc", run relay, shutdown. | `/usr/src/usr.sbin/relayd/proc.c:180` | **REFERENCE** | L | relayd's proc_init/proc_setup is a full privsep framework (struct privsep, instance arrays, IMSG_CTL_PROCFD fd-passing). Adopting it wholesale is heavy and libevent-coupled; fugu's flat 4-worker model is simpler and intentionally foreground. Mirror the ordering/teardown discipline, don't link. | proc_init at /usr/src/usr.sbin/relayd/proc.c:180 is the architectural analogue (setup pipes -> exec children -> connect). REFERENCE only: it's program-internal (usr.sbin), not a linkable lib, and pulls in libevent + the privsep struct model that fugu deliberately avoids. | +| spawn | socketpair + fork; in child dup2 the pair onto fd 3, redirect non-ui stdio to /dev/null, closefrom, execv; in parent ip_init the channel. | `/usr/src/usr.sbin/relayd/proc.c:70` | **REFERENCE** | M | proc_exec at proc.c:70 is the canonical base pattern: dup2 the pipe onto a fixed PROC_PARENT_SOCK_FILENO then execvp. fugu mirrors it (fd 3 = IMSG_CHILD_FD). It's not linkable, so REFERENCE: align edge-case handling (fd==target fcntl(F_SETFD,0) vs dup2) but keep fugu's own closefrom(3)/dev-null hardening, which is already strong. | closefrom(2) confirmed /usr/include/unistd.h:491 and already used. fugu's spawn is arguably tighter than proc_exec (closefrom(IMSG_CHILD_FD+1) drops every stray fd; proc_exec relies on F_SETFD discipline). Mirror, don't replace. | +| delegate_net / delegate_exec / delegate_fetch (config+secret delegation group) | Build a JSON config object per worker and ip_compose+ip_flush M_CONFIG, then resolve and send M_SECRET, freezero-ing the secret copy. | imsg_compose(3) | **KEEP** | S | the imsg send path is already base (imsgproto wraps imsg_compose/imsgbuf_flush); JSON-config building is app logic | Secret hygiene already correct: freezero(key,strlen) confirmed in malloc(3); secrets travel over imsg payload, never argv/env (charter-compliant). No base helper builds the JSON config. Keep; this is the secrets-never-in-argv design working as intended. | +| resolve_secret | Return an inline secret (xstrdup) or read+trim the first line of a key file via getline. | getline(3) | **KEEP** | S | getline + trim is the base idiom; no secret-file reader in base | Reading a credential file's first line is app policy. getline already used. Caller freezero's the result. Keep. | +| worker_by_ep | Map an endpoint id to its active worker slot (linear scan of 4). | — | **KEEP** | S | 4-element routing table lookup; no library seam | Trivial routing helper over fugu's own WDESC table. Nothing in base applies. | +| relay_loop | poll() over all active worker fds; on POLLIN drain frames, on POLLHUP/ERR exit. The coordinator's central event loop. | `/usr/src/sbin/dhcpleased/frontend.c:167` event(3) | **REFERENCE** | L | libevent IS base (/usr/lib/libevent.so, event(3)) and dhcpleased/frontend.c:167-190 shows the canonical event_init()+per-channel event_set/event_add+event_dispatch() design. But for a fixed set of 4 channels with no timers/signals beyond SIGPIPE, a flat poll() loop is simpler, has no callback indirection, and keeps the coordinator foreground/non-daemon (fugu's charter). RECOMMEND evaluating libevent only if timers/backpressure/more channels arrive; do NOT adopt now. | Honest tradeoff: this is the big design question. event_dispatch confirmed in event(3) and libevent.a/.so present in base. dhcpleased frontend(): event_init() (frontend.c:167), event_set/event_add per imsg channel (185-187), event_dispatch() (190). The win would be write-side backpressure (imsgbuf_queuelen -> EV_WRITE, see relayd proc.c imsg_event_add) which fugu's poll loop currently doesn't arm (it only POLLINs; relies on ip_flush blocking). That's the one real gap worth noting — but addressable in the poll model too. REFERENCE, recommend-not-adopt. | +| relay_drain | imsgbuf_read one worker, loop ip_frame_get pulling raw frames, route each by imsg_get_id to the destination worker via ip_forward+ip_flush, never reassembling payloads. | `/usr/src/usr.sbin/relayd/proc.c:593` imsg_forward(3) | **REFERENCE** | M | fugu already uses the exact modern base primitives (imsgbuf_read, imsg_get, imsg_get_id, imsg_forward, imsg_free — all confirmed in /usr/include/imsg.h via the imsgproto wrapper). proc_dispatch (proc.c:593) is the structural twin: imsgbuf_read -> for(;;) imsg_get -> handle/forward -> imsg_free, then imsg_event_add. Can't link it (program-internal + libevent-bound), so REFERENCE: mirror its EPIPE/n==0 channel-death handling. | imsg_forward confirmed /usr/include/imsg.h:162, imsg_get_id at :158, imsgbuf_read at :144 — fugu is already on the post-rewrite imsgbuf_* API (the rewrite the lead warned about), so no migration debt. The relay's forward-by-id-without-reassembly is precisely what imsg_forward(3) is for and what relayd/proc_forward_imsg does. Strongest alignment point: copy proc_dispatch's death/EPIPE branch semantics. | +| shutdown_workers | Close each worker channel (EOF -> child exits), clear imsg state, then waitpid each pid. | `/usr/src/usr.sbin/relayd/proc.c:351` waitpid(2) | **REFERENCE** | S | proc_kill (proc.c:351) does the analogous close-pipes-then-reap; fugu's two-pass (close all, then wait all) is correct and arguably cleaner. No linkable helper; align the reaping discipline only. | waitpid(2) base. fugu's ordering (EOF every child first so they all exit concurrently, then reap) avoids serial-shutdown stalls. Keep the implementation; REFERENCE proc_kill purely for the close->SIGsomething->wait convention if a forced-kill timeout is ever wanted (fugu currently relies on EOF, which is fine for cooperative workers). | + +### `src/net/net.c, src/net/net_run.c` + +_net.c is a textbook base-idiomatic worker main (getopt/log/pledge/fstat-S_ISSOCK) — fully aligned, nothing to ADOPT beyond what it already uses; net_run.c is app-specific agent-loop/imsg-dispatch/blocking-HTTPS logic that is mostly KEEP, with the imsg plumbing and synchronous TLS HTTP client being REFERENCE alignments to base imsgbuf_* and ftp(1)'s url_get/tls_read blocking client._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| main (net.c) | Worker entry: getopt(d/v), log_init/log_setverbose, ignore SIGPIPE, fstat the inherited fd and require S_ISSOCK, ip_init the imsg channel, run net_loop. | `/usr/src/usr.sbin/httpd/httpd.c (getopt main idiom); /usr/include/sys/stat.h:140 (S_ISSOCK)` getopt(3) | **REFERENCE** | S | Already mirrors base worker-main conventions verbatim (getopt loop, SIG_IGN on SIGPIPE per ssh mux.c:1991, log_init/fatal from the copied base log.c, S_ISSOCK guard); nothing to change, keep aligned. SIG_IGN-on-SIGPIPE matches base; could use ssh_signal-style wrapper but that is ports/ssh-internal, not warranted. | All primitives (getopt, signal, fstat, S_ISSOCK, fprintf, __progname) are pure libc/base. The only non-libc calls are fugu's own log_*/ip_init wrappers, which themselves wrap base log.c and base imsg. This main is the idiomatic OpenBSD privsep-worker pattern; no ADOPT gap. | +| net_loop | The imsg event loop: emit M_READY, then ip_get/ip_read-drain dispatch on M_SECRET/M_CONFIG/M_SUBMIT/M_SUMMARIZE/M_SHUTDOWN; stores the API key via xmalloc/freezero; narrows pledge to 'stdio rpath inet dns' on entry. | `/usr/src/lib/libutil/imsg.c; /usr/src/lib/libutil/imsg_init.3` imsg_init(3) | **REFERENCE** | M | The read-drain-then-dispatch shape is the canonical imsg consumer loop; fugu reimplements it behind ip_get/ip_read which wrap the current imsgbuf_* API (imsgbuf_init/imsgbuf_read/imsg_get) — confirmed current via imsg_init(3) listing imsgbuf_init/imsgbuf_read/imsgbuf_flush. Align idioms to base; the dispatch switch itself is app-specific (KEEP-flavoured). | Verified base now exposes imsgbuf_init/imsgbuf_read/imsgbuf_flush/imsg_get (the post-rewrite API) in imsg_init(3) and /usr/include/imsg.h. The actual imsgbuf_* adoption lives in fugu's ip_* wrapper (imsgproto.c, not in these files); these files only consume that wrapper. pledge/freezero are pure base. Loop body (secret handling, turn/summarize routing) is application logic = KEEP, but the plumbing is a base-aligned REFERENCE. | +| net_run_turn | The agent loop: repeatedly call the Messages API; while the model requests tools, build the assistant message, run tools, append tool_results, and re-call, up to NET_MAX_ITERS (50). | — | **KEEP** | L | Anthropic agent/tool-use orchestration; no base analogue. | Pure application control flow over fugu's own buf/json/imsg helpers. The synchronous structure (block on each API call, block on each tool round) is the deliberate design; base has nothing comparable. Only the inner net_api_call has a base REFERENCE. | +| net_summarize | /compact: append a closing user turn, run one tool-less Messages call with the summary system prompt (deltas hushed), emit collected text as M_SUMMARY. | — | **KEEP** | M | Anthropic-specific summarisation call; no base equivalent. | Application logic; uses fugu json_writer + net_api_call. No base seam. | +| net_api_call | One streamed Messages call: build JSON body + HTTP request, tlsconn_connect, write_all the request, then loop tlsconn_read into a 4KB buffer feeding http_resp_push/eof, capture non-200 error body, classify status. | `/usr/src/usr.bin/ftp/fetch.c:306 (url_get); /usr/src/usr.bin/ftp/fetch.c:1773 (stdio_tls_read_wrapper)` tls_connect(3) | **REFERENCE** | M | libtls itself is base (-ltls, tls.h) and IS used via fugu's tlsconn wrapper; but the request-build / status-parse / blocking-read-loop is solved only inside ftp(1) (usr.bin), not a linkable library, so REFERENCE. Mirror ftp(1)'s blocking idiom: retry tls_read on TLS_WANT_POLLIN/POLLOUT (fetch.c:1773-1782) rather than treating a short/again read as fatal, and its status-line/header parsing in url_get. | Confirmed ftp(1) does exactly fugu's pattern: tls_connect_socket then a do/while tls_read loop that retries on TLS_WANT_POLLIN/TLS_WANT_POLLOUT (stdio_tls_read_wrapper, fetch.c:1773), wrapped through funopen at fetch.c:666; url_get (fetch.c:306) builds the request and parses the response status. fugu's tlsconn_read presumably already handles TLS_WANT; the alignment value is in the edge-case/status-parsing handling. No linkable HTTP client in base = not ADOPT. | +| conv_append | Append one JSON message object to a messages array that ends in ']' (drop ']' , add ',' + object + ']'). | — | **KEEP** | S | Trivial JSON-array splice on fugu's buf; no base API. | Manipulates fugu buf directly. No base JSON facility (jsmn is parse-only and vendored). | +| build_assistant_msg / build_tool_results (assistant+tool_result JSON builders) | Serialize the collected turn into Anthropic message JSON: an assistant message (text + tool_use blocks) and a user message of tool_result blocks (the latter runs each tool and embeds its content/is_error). | — | **KEEP** | M | Anthropic message-shape serialization via fugu json_writer; no base JSON writer. | Cohesive pair that builds the two conversation-turn JSON objects. Pure application logic; base has no JSON-emitter library (jsmn is read-only). | +| send_tool_and_wait | Compose+flush an M_TOOL_REQUEST to the tool worker, then block ip_get/ip_read until the matching M_TOOL_RESULT (or M_SHUTDOWN), parse content/is_error, ignoring other messages mid-turn. | `/usr/src/lib/libutil/imsg.c` imsg_init(3) | **REFERENCE** | M | The compose/flush then read-until-reply pattern is the base imsg RPC idiom (imsg_compose + imsgbuf_flush + imsgbuf_read/imsg_get). fugu wraps it as ip_compose/ip_flush/ip_get/ip_read; align to current imsgbuf_* naming and MAX_IMSGSIZE bounds. The synchronous-block-mid-turn policy is app-specific. | Uses fugu ip_* wrappers (which sit on base imsgbuf_*). Verified base API names current in imsg_init(3). Not ADOPT because the wrapper and the blocking-RPC policy are fugu's; the underlying transport is the base imsg adoption already in the ip_* layer. | +| tool_dst | Route a tool name to its worker endpoint: web_search/web_fetch -> EP_FETCH, else EP_EXEC. | — | **KEEP** | S | Two strcmp dispatch; pure fugu routing policy. | Trivial; no base seam. | +| turncollect_free | Free a turncollect: buf_free text, free each toolcall id/name/input, free arrays and stop_reason. | — | **KEEP** | S | Struct destructor over fugu allocations; no base API. | Standard free helper. free(3) is libc but there is no base 'destructor' to adopt. | +| net_emit / net_emit_error | Emit an imsg (ip_compose+ip_flush, fatalx on failure); net_emit_error formats 'what: detail' into a buf and emits M_ERROR to the UI. | `/usr/src/lib/libutil/imsg.c` imsg_init(3) | **REFERENCE** | S | Thin convenience over the base imsg send path (compose+flush). Keep, but ensure the wrapped calls track current imsgbuf_flush semantics. net_emit_error formatting is app-specific. | Wraps fugu ip_* over base imsgbuf_*. fatalx on send failure is the base daemon convention (log.c). | +| net_apply_config / cfg_str | Parse a JSON config object (json_parse) and update model/max_tokens/system/host/port/cafile; max_tokens via strtonum(1,1000000); cfg_str fetches a JSON string field. | strtonum(3) | **ADOPT** | S | Already correctly uses strtonum(3) (base, ) with min/max + errstr for max_tokens — the idiomatic OpenBSD safe-integer parse; no change needed, this is the win already taken. The JSON field extraction (json_obj_get/json_tok_strdup) is fugu/jsmn and stays KEEP. | strtonum(3) confirmed in base (man 3 strtonum, declared in ). This is a genuine base-API adoption fugu already does correctly. The surrounding JSON config plumbing has no base equivalent (jsmn vendored). | +| net_ensure_ready | One-time readiness: load the TLS client config from cafile (tlsconn_client_config), then narrow pledge to 'stdio inet dns' and set ready. | tls_connect(3) | **REFERENCE** | S | The privilege-narrowing-after-setup (open CA bundle under rpath, then drop rpath) is the canonical pledge staging idiom. CA load uses libtls (base) behind tlsconn_client_config. Keep; align CA-load error handling to libtls conventions. | pledge(2) and libtls are base. The staged pledge narrowing matches base daemons. tlsconn_client_config is fugu's wrapper over base tls_config_*; the adoption already happened in the tlsconn layer (not these files). | +| net_state_clear | Teardown: freezero the key, free model/system/host/port/cafile, tls_config_free, zero the struct. | freezero(3) | **ADOPT** | S | Correctly uses freezero(3) (base, malloc(3) family) to scrub the secret key and tls_config_free(3) (base libtls) — both idiomatic; no change needed. Honors the charter's 'secrets never linger' intent. | freezero(3) confirmed in MALLOC(3) listing; tls_config_free is base libtls. Both are real base symbols already adopted. The rest is plain free(3). | +| net_body | HTTP body sink trampoline: on status 200 feed bytes to the SSE parser, else accumulate into the error buf. | — | **KEEP** | S | Callback trampoline between fugu http_resp and sse parser; no base API. | Pure fugu pipeline glue. | +| sse_to_anthropic | SSE-event trampoline: forward (event,data,len) to anthropic_stream_event. | — | **KEEP** | S | One-line forwarder between fugu sse and anthropic parsers; no base seam. | Server-Sent-Events is not a base facility; vendored/own parser. KEEP. | +| anthropic stream callbacks (on_text/on_tool_call/on_usage/on_stop/on_err/on_done) | Six libcb callbacks that relay streamed events to the UI (M_DELTA/M_TOOL_CALL/M_USAGE/M_ERROR) and collect them into the active turncollect; on_done is a no-op. | — | **KEEP** | M | Anthropic streaming event handlers; no base analogue. | Cohesive callback set wired into NET_CBS. They emit imsg via net_emit (base-imsg REFERENCE already covered there) but the event semantics are entirely Anthropic-specific. KEEP. | + +### `src/exec/exec.c, src/exec/exec_run.c` + +_Thin worker entry + cwd-confined sandbox; the unveil/pledge/execpromises sandbox is an idiomatic base privsep pattern (REFERENCE), getopt/signal are libc (ADOPT, already used), the rest is app-specific imsg/tool/json plumbing (KEEP)._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| main | fugu-exec worker entry: getopt("dvN"), log_init/setverbose, ignore SIGPIPE, fstat the imsg fd 3 to refuse direct invocation, then ip_init + exec_sandbox + exec_loop. | getopt(3) | **ADOPT** | S | Already on the base idiom; nothing to change. | Pure base-idiom entry: getopt(3) option loop, signal(SIGPIPE,SIG_IGN) (cf /usr/src/usr.bin/ftp/ftp.c), fstat(2)+S_ISSOCK guard, __progname libc extern. The only non-base calls (log_init, ip_init) are fugu-internal helpers and stay KEEP by composition, but main() itself is textbook libc. Verified signal/SIGPIPE idiom present in base (ftp.c, dig). | +| exec_sandbox | Builds the cwd-confined no-network sandbox: unveil(".","rwc"), hide ~/.fugu, unveil base tool/lib dirs rx, optional resolver/trust store for opt-in subprocess net, unveil(NULL,NULL) lock, then two-arg pledge(promises, execpromises) where execpromises is the ceiling inherited across execve(2) for ksh/grep/find/ls children — inet-excluding unless allow_subprocess_net. | `/usr/src/libexec/ld.so/ldd/ldd.c:186` pledge(2), unveil(2) | **REFERENCE** | S | No base library wrapper to link — pledge/unveil are raw syscalls; this is align-the-idiom, not adopt-a-symbol. | Confirmed already idiomatic. The exec->execpromises pattern (tighten a child's pledge ceiling before/around execve) is exactly ldd's pledge("stdio rpath exec","stdio rpath") at /usr/src/libexec/ld.so/ldd/ldd.c:186, verified by reading lines 185-186. Two-arg pledge(promises,execpromises) confirmed in pledge(2) synopsis (man line 10) and behavior at man lines 218/286. unveil-then-lock(NULL,NULL) before the final pledge matches unveil(2)'s 'strongly recommended to lock' guidance. Lead's 'sandbox idiom is REFERENCE' confirmed correct; structure is sound. | +| unveil_or_skip | Helper: unveil(path,perms) tolerating ENOENT so optional dirs (/sbin, /usr/local/lib, resolver files) need not exist. | `/usr/src/sbin/sysctl/sysctl.c:270` unveil(2) | **REFERENCE** | S | No linkable helper in base — every program open-codes the same 2-line guard. | Byte-for-byte the base idiom: /usr/src/sbin/sysctl/sysctl.c:270 does `if (unveil(_PATH_DEVDB,"r")==-1 && errno!=ENOENT)` (also lines 272, 275), verified by grep. fugu's wrapper is the correct factoring of that pattern; nothing to adopt, just confirms idiomatic alignment. | +| exec_loop | imsg receive loop: ip_get/ip_read drain, dispatch M_CONFIG->apply_config, M_TOOL_REQUEST->handle_tool, M_SHUTDOWN->exit; frees each msg. | — | **KEEP** | S | app-specific dispatch over fugu's imsgproto wrapper and fugu_msg taxonomy. | Underlying transport already rides base imsg via fugu's imsgproto, but the loop control flow and message types (M_CONFIG/M_TOOL_REQUEST/M_SHUTDOWN) are fugu's. No base equivalent for the dispatch itself. | +| apply_config | Parse M_CONFIG JSON object and read allow_write bool into execstate. | — | **KEEP** | S | app-specific config schema via fugu's vendored jsmn wrapper; no base JSON in charter scope. | Trivial schema-specific parse over json_doc; out of scope for base adoption (jsmn is the only allowed vendored dep). | +| handle_tool | Dispatch M_TOOL_REQUEST by tool name (read/write/edit/shell/grep/find/ls) into tool_* implementations, marshal missing-arg errors, then emit_result. | — | **KEEP** | M | core app tool registry; no base equivalent. | Lead confirms tool dispatch is app-specific KEEP. The strcmp-chain could be a dispatch table but that's a style nit, not a base-adoption opportunity. Arg extraction via obj_getstr, error strings via buf. | +| emit_result | Build M_TOOL_RESULT JSON {id,content,is_error} via fugu's json_writer and send it with ip_compose/ip_flush to EP_NET. | — | **KEEP** | S | app-specific result framing over fugu's JSON writer and imsg wrapper. | No base option: JSON output is fugu's own writer; imsg compose/flush is fugu's imsgproto over base imsgbuf_*. | +| obj_getstr | Convenience accessor: fetch a JSON string value by key from an object token, strdup'd, else NULL. | — | **KEEP** | S | thin accessor over fugu's vendored jsmn json_doc; no base JSON library in scope. | Pure jsmn-wrapper helper (json_obj_get + json_tok_type + json_tok_strdup); nothing in base to align to. | + +### `src/exec/tools_file.c, src/exec/tools_shell.c` + +_Both files already lean on base libc primitives (mkstemp, memmem, closefrom) correctly; the composite helpers (atomic-write, file-slurp, fork+exec+poll capture) have no LINKABLE base equivalent — their closest base analogues are program-internal (acme-client/sed atomic-write, ssh subprocess capture), so they are REFERENCE/KEEP, not ADOPT._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| tool_read | Public entry: slurp a file into out, formatting an errno message on failure. | — | **KEEP** | S | no base option: app-specific buf+errno-message glue, no linkable libc equivalent for 'read whole file into growable buf' | Just dispatches to static slurp() and renders strerror into the struct buf. Nothing for base to replace; the real reuse opportunity lives in slurp(). | +| tool_write | Public entry: allow_write policy gate, then durable atomic write of content to path. | — | **KEEP** | S | no base option: the allow_write authorization gate and message are fugu policy; the durable-write mechanics are in write_atomic() | Charter-relevant guard (allow_write no). Base has no equivalent for the policy decision; mechanics evaluated under write_atomic. | +| tool_edit | Read file, locate a unique old_string, splice in new_string, atomic-write the result. | `/usr/include/string.h:122` memmem(3) | **ADOPT** | S | Already adopted: the load-bearing search uses base memmem(3) (libc), declared at string.h:122, confirmed binary-safe via big_len; the surrounding splice is app logic. | memmem(3) is genuine base libc (man 3 memmem; string.h:122). fugu already calls it correctly for both find and the second-occurrence uniqueness probe (m+1, content.len-off-1). The splice/orchestration itself stays fugu's; no further base API applies. Marked ADOPT because the one library-replaceable seam (substring search) is already on the base API and should stay there rather than hand-rolling. | +| write_all | Write a full buffer to an fd, looping on partial writes and retrying EINTR. | `/usr/src/usr.bin/ssh/atomicio.c:50` | **REFERENCE** | S | ssh's atomicio()/atomiciov() is the canonical full-write loop but lives in ssh's tree, not libc/libutil; grep of /usr/include shows no atomicio declaration, so it is not linkable. | Verified: 'grep atomicio /usr/include' returns nothing — atomicio is per-program in usr.bin/ssh, so KEEP-by-link but mirror its EINTR/EAGAIN handling. fugu's loop already matches the EINTR-retry + partial-write accounting; the only gap vs atomicio is it does not treat a 0-byte write as an error. Low value to change; cite ssh/atomicio.c as the idiom of record. | +| slurp | Read an entire file into a growable buf with EISDIR/EFBIG guards and a 1 MiB cap. | `/usr/src/usr.bin/ssh/sshbuf-io.c` | **KEEP** | S | no base option: base has no libc/libutil 'load entire file into buffer' call; grep of /usr/include finds no read_file/load_file, and sshbuf_load_file is ssh-internal | Confirmed no linkable equivalent (no read_file/load_file in /usr/include; sshbuf_load_file only under usr.bin/ssh). The S_ISDIR->EISDIR pre-check and TOOLS_MAXFILE cap are fugu-specific safety. open/fstat/read-loop is the right base primitive set already. Keep as-is. | +| write_atomic | Durable atomic file replace: mkstemp sibling temp, write, preserve mode via fchmod, fsync, rename; unlink temp on any failure. | `/usr/src/usr.sbin/acme-client/fileproc.c:59` mkstemp(3) | **REFERENCE** | M | The base atomic-write pattern (mkstemp -> fchmod -> write -> close -> rename, unlink-on-fail) exists only inside programs (acme-client serialise(), sed, cron crontab.c), not as a linkable libc helper; fugu already uses the base mkstemp(3) primitive correctly. | Canonical base site: acme-client/fileproc.c serialise() at fileproc.c:59 (mkstemp), fchmod fileproc.c:63, rename fileproc.c:77, unlink-on-fail fileproc.c:87 — but NO fsync. sed/main.c:405 preserves mode via 'fchmod(fd, sb.st_mode & ALLPERMS)', matching fugu's st_mode & 0777 preservation; cron/crontab.c:398/525 same shape. fugu IMPROVES on all of them by adding fsync(2) before rename for crash durability — keep that. Action: align edge cases to acme-client (it is the idiom of record) but do not regress fsync. mkstemp(3) itself is confirmed base libc (stdlib.h). | +| run_capture | Fork a child with stdin=/dev/null and stdout/stderr to a pipe, closefrom() to drop the imsg fd, execv, poll a CLOCK_MONOTONIC deadline, cap captured output, SIGKILL on timeout/overflow, reap with EINTR-safe waitpid. | `/usr/src/usr.bin/ssh/misc.c:2786` closefrom(2) | **REFERENCE** | L | ssh/misc.c subprocess() is a near-identical pipe+fork+dup2(/dev/null)+closefrom(STDERR_FILENO+1)+execv capture, but it is an ssh-internal helper (not libutil) and stops at returning the pipe fd — it has no timeout or output cap, both of which fugu adds. popen(3) is unusable (no stderr capture, no timeout). So overall KEEP-by-link, REFERENCE-by-idiom. | Strong analogue verified: ssh/misc.c:2786 pipe(p); 2796 fork(); 2823 dup2(devnull,STDIN); 2834 dup2(fd,STDOUT); 2838 closefrom(STDERR_FILENO+1); 2863 execve/execv; parent waitpid EINTR loop ~2881 — mirrors fugu line-for-line including the closefrom(STDERR_FILENO+1) to shed inherited fds (fugu's comment: drop imsg fd). closefrom(2) is genuine base (unistd.h:491). The CLOCK_MONOTONIC poll-deadline is the ssh monotime()/ptimeout idiom (misc.c:1735, 341). Differences worth keeping in fugu: wall-clock timeout via monotonic deadline, TOOLS_MAXOUTPUT cap with SIGKILL backpressure, truncation/timeout annotations. Align error/EINTR handling to ssh; do not adopt popen. | +| tool_shell / tool_grep / tool_find / tool_ls | Thin public wrappers that build a fixed argv (ksh -c, grep -RnI, find, ls -la) and delegate to run_capture. | — | **KEEP** | S | no base option: these are fixed-argv constructors selecting base programs (/bin/ksh, /usr/bin/grep, /usr/bin/find, /bin/ls); there is no linkable library form, and re-implementing grep/find/ls in-process would violate simplicity and the sandbox model | Tight cohesive group of 4 trivial argv-builders over run_capture. Correct design: shell out to base tools and inherit the worker's unveil/pledge sandbox rather than re-implement. The (char*)(uintptr_t) casts launder const into the execv char*const[] contract — harmless. Nothing for base to provide here; the only shared mechanism (capture) is covered by run_capture above. | + +### `src/fetch/fetch.c, src/fetch/fetch_run.c, src/fetch/kagi.c` + +_Fetch worker: app-specific imsg/tool plumbing (KEEP) over base libtls (tls(3) already ADOPTED via direct calls); the connect+SSRF dial loop and TLS handshake loop mirror ftp(1)'s program-internal fetch.c (REFERENCE); getaddrinfo(3) usage is correct libc; addr_blocked SSRF guard and Kagi formatting have no base equivalent (KEEP)._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| main | fugu-fetch worker entry: getopt, log_init, SIGPIPE ignore, fstat-asserts imsg fd 3 is a socket, ip_init, runs fetch_loop. | getopt(3) | **KEEP** | S | App-specific worker bootstrap; already built from base primitives (getopt/signal/fstat) with nothing to swap. | The fd-3-is-a-socket guard via fstat+S_ISSOCK is idiomatic OpenBSD; no helper to adopt. | +| fetch_loop | Main imsg event loop: pledge('stdio rpath inet dns'), send M_READY, dispatch M_SECRET/M_CONFIG/M_TOOL_REQUEST/M_SHUTDOWN, freezero the Kagi token. | pledge(2) | **KEEP** | S | App-specific brokered-tool loop over fugu's imsg layer; pledge is already used correctly. | Secret handling (freezero on token, never argv/env) matches charter. Nothing linkable to adopt. | +| apply_config | Parse M_CONFIG JSON, override cafile/kagi_host/kagi_port/allow_private. | — | **KEEP** | S | App config over fugu's json wrapper; no base JSON library by charter. | Pure application glue. | +| ensure_ready | Lazily build TLS client config from cafile via tlsconn_client_config, then narrow pledge to 'stdio inet dns'. | tls_load_file(3) | **KEEP** | S | Already sits on base libtls via fugu's tlsconn wrapper (out of this file's scope); the privilege-narrowing pledge step is app-specific. | Confirmed tls_config_set_ca_file is base (TLS_LOAD_FILE(3)). The wrapper itself lives in tlsconn.c, not audited here; this function is just the call+pledge orchestration => KEEP. | +| handle_tool | Parse M_TOOL_REQUEST JSON (id/name/input), dispatch web_search vs web_fetch, emit M_TOOL_RESULT. | — | **KEEP** | S | App-specific tool router over fugu json/imsg; no base analog. | Application dispatch logic. | +| do_web_search | Build Kagi search path+auth header, call http_fetch (guard off for trusted Kagi host), format results; freezero the auth header holding the token. | — | **KEEP** | S | Kagi-API-specific request orchestration; no base equivalent. | Good: token-bearing auth value is freezero'd after use, consistent with charter. | +| do_web_fetch | Parse URL, enforce https-only, http_fetch with SSRF guard (unless allow_private), reject redirects, branch on content-type to html2text/text/binary with caps. | — | **KEEP** | M | App-specific fetch policy (no-redirect, content-type handling, size caps); no base helper. | strcasestr/strncasecmp are libc and used correctly. The policy (no redirect follow to avoid re-guarding) is deliberate and not something base offers. | +| ms_left | Compute milliseconds remaining to a CLOCK_MONOTONIC deadline, clamped to [0, FETCH_TIMEOUT*1000]. | `/usr/src/usr.bin/ftp/util.c:1081` clock_gettime(2) | **KEEP** | S | Tiny deadline-math helper; clock_gettime already base, no library function computes a clamped ms remainder. | Could optionally use timespecsub(3) from for the subtraction, but the manual arithmetic is fine and idiomatic; not worth a finding upgrade. | +| io_wait | poll() a single fd for events until the deadline; 1 ready / 0 timeout / -1 error. | poll(2) | **KEEP** | S | Thin single-fd poll wrapper around base poll(2); nothing to adopt. | libevent is base and allowed, but a blocking deadline-bounded worker doing one request at a time correctly uses poll directly; event-loop migration would be overkill here. | +| addr_blocked | SSRF guard: classify a resolved sockaddr as loopback/RFC1918/link-local/CGNAT/multicast/ULA/v4-mapped and refuse non-public addresses. | — | **KEEP** | M | No base option: confirmed no private-range/SSRF helper in /usr/src/lib/libc (grep empty); base ftp(1) has no such guard at all. | Uses base IN6_IS_ADDR_LOOPBACK/LINKLOCAL/MULTICAST/UNSPECIFIED/V4MAPPED macros correctly. The IPv4 octet checks and fc00::/7 ULA + CGNAT (100.64/10) handling are hand-rolled because base provides no equivalent. This is the security core and must stay in fugu. | +| dial | getaddrinfo(host,port) then iterate results: SSRF-guard each resolved addr, non-blocking connect with EINPROGRESS+io_wait deadline, return connected fd. | `/usr/src/usr.bin/ftp/fetch.c:599` getaddrinfo(3) | **REFERENCE** | M | getaddrinfo itself is correct libc usage (cf. /usr/src/usr.bin/ftp/fetch.c:531); the connect-with-timeout loop should align to ftp's timed_connect, which is a program-internal helper (defined /usr/src/usr.bin/ftp/util.c:1081, declared in ftp's extern.h) — not linkable, so REFERENCE not ADOPT. | ftp/fetch.c:599 calls timed_connect(fd,res->ai_addr,res->ai_addrlen,connect_timeout) while iterating getaddrinfo res — same shape as dial(). ftp uses its own non-blocking-connect helper; mirror its edge-case handling (per-ai fallthrough, SO_ERROR check). fugu adds the addr_blocked guard per-ai which ftp lacks; keep that. getaddrinfo(3) confirmed declared netdb.h:316, implemented lib/libc/asr/getaddrinfo.c. | +| http_fetch | One HTTPS GET under a wall-clock deadline: dial, build Host/UA/Accept headers + extras, tls_client/tls_configure/tls_connect_socket, deadline-bounded handshake/write/read loops, push into http_resp, cap body. | `/usr/src/usr.bin/ftp/fetch.c:646` tls_connect_socket(3) | **REFERENCE** | M | The libtls calls ARE the base API and are already adopted (drop-in); what to mirror is the TLS_WANT_POLLIN/POLLOUT handshake/read loop idiom from ftp(1)'s program-internal fetch.c. The HTTP request build, deadline plumbing, and body cap are fugu-specific. | ftp/fetch.c:646-666 does the identical tls_client->tls_configure->tls_connect_socket->tls_handshake-loop sequence; fugu correctly extends it with TLS_WANT_POLLIN/OUT + io_wait for non-blocking deadline enforcement (ftp uses funopen/blocking stdio instead). Confirmed tls_connect_socket in TLS_CONNECT(3), tls_config_set_ca_file in TLS_LOAD_FILE(3). No drop-in to swap — already on base libtls; align idioms/error-string handling to ftp's sockerror(). | +| fetch_body | http_resp body sink: append decoded bytes into struct buf up to a cap, set truncated flag. | — | **KEEP** | S | Callback over fugu's buf module enforcing the download cap; no base analog. | Application size-capping callback. | +| emit_result | Serialize {id,content,is_error} via fugu json_writer and ip_compose/ip_flush an M_TOOL_RESULT. | — | **KEEP** | S | App-specific result serialization over fugu's json/imsg layers. | Pure glue. | +| obj_getstr | Helper: fetch a string-typed value for a key from a fugu json object, strdup'd. | — | **KEEP** | S | Thin accessor over fugu json; no base JSON API by charter. | Duplicated logic with kagi.c:field() but local to this file; trivial. | +| state_clear | Teardown: freezero the Kagi token, free cafile/host/port, tls_config_free, zero the struct. | freezero(3) | **KEEP** | S | App-specific cleanup already built from base freezero + libtls free; nothing to adopt. | Correct secret hygiene: freezero on the token rather than free. | +| kagi_search_path | Build the Kagi /api/v0/search query path with url_encode'd query and limit=10. | — | **KEEP** | S | Kagi-endpoint-specific URL builder; no base equivalent. | url_encode is a fugu module (url.h), out of scope here. | +| kagi_format_results | Parse Kagi JSON response: handle error envelope, walk data[] array, extract title/url/snippet of search-type results, field-cap and format up to KAGI_MAX_RESULTS. | — | **KEEP** | M | Kagi-API-specific response formatting; no base analog. | Per-field caps (KAGI_TITLE/URL/SNIPPET_MAX) defend the model against a hostile result flooding context — app policy, must stay. | +| field | Helper: fetch a string-typed value for a key from a fugu json object, strdup'd. | — | **KEEP** | S | Thin json accessor; identical to fetch_run.c:obj_getstr but file-local static. | Could be de-duplicated with obj_getstr into a shared json helper, but that is a fugu-internal refactor, not a base-adoption finding. | + +### `src/ui/ui_curses.c` + +_Wide-char curses TUI on a poll(2) loop: the curses/libc primitives (get_wch, mvaddnwstr, wcwidth, mbrtowc, wcrtomb, setlocale, nl_langinfo, resizeterm) are all base-linkable and already used idiomatically — verdict REFERENCE/ADOPT to align with ssh/utf8.c and mandoc/term_ascii.c; the transcript/block model, line editor, imsg pump and slash-command handling are app-specific KEEP._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| ui_curses_run | Top-level TUI: locale/UTF-8 setup, unveil/pledge narrowing, initscr+raw/noecho/keypad/nodelay, SIGWINCH sigaction, and the poll(2) loop multiplexing stdin (get_wch) and the imsg fd. | `/usr/src/usr.bin/mandoc/term_ascii.c:93` curs_initscr(3) | **REFERENCE** | M | curses init/raw-mode setup is base-linkable and already idiomatic; the locale forcing mirrors mandoc term_ascii.c:93-94 setlocale(LC_CTYPE,"")->"en_US.UTF-8". The poll(2)+curses+imsg event multiplexing is fugu-specific (no base curses program drives a non-blocking get_wch poll loop), so align idioms only, do not adopt a driver. | setlocale(LC_CTYPE,"") then nl_langinfo(CODESET)!="UTF-8" fallback to en_US.UTF-8/C.UTF-8 is exactly mandoc's UTF8_LOCALE idiom (term_ascii.c:37,93-94) and ssh utf8.c's CODESET test (utf8.c:50-60). nodelay+get_wch drain loop is correct for non-blocking wide input. Consider libevent (base) instead of hand-rolled poll, but charter-neutral; keep poll. | +| on_winch | SIGWINCH handler: sets a volatile sig_atomic_t flag so the poll loop can resize. | sigaction(2) | **KEEP** | S | trivial async-signal-safe flag setter; no base library equivalent to adopt | Idiomatic: only touches a volatile sig_atomic_t. sa_flags 0 (no SA_RESTART) is deliberate so poll(2) returns EINTR — matches base practice. | +| block_new / blocks_free / replay_block | Transcript model: grow the block array, free it, and convert a replayed convo part into a typed block. | — | **KEEP** | S | application-specific transcript/block storage; no base equivalent | Uses project xreallocarray/buf helpers (reallocarray(3) is the base primitive underneath, already covered elsewhere). Pure app logic. | +| prefix_of / attr_of | Map a block kind to its display prefix string and curses attribute (A_BOLD/A_DIM/etc). | curs_attr(3) | **KEEP** | S | app styling table; the A_* constants are base but the mapping policy is fugu-specific | A_BOLD/A_DIM are base curses attrs (curs_attr(3)); only the kind->attr policy is app-specific. Nothing to adopt. | +| cw | Display width of a wide char, clamping control/unknown (wcwidth==-1) to 1 column. | `/usr/src/usr.bin/ssh/utf8.c:146` wcwidth(3) | **ADOPT** | S | already a thin wrapper over the base wcwidth(3); keep wrapper but align the -1 fallback to base practice | wcwidth(3) is base libc (WCWIDTH(3), declared ). ssh/utf8.c:146 and less/line.c:256 handle wcwidth==-1 explicitly; fugu's clamp-to-1 is a reasonable display choice. Already adopting the base API — this is a correct wrapper, not a reimplementation. | +| ascii_rep / fold_wcs | Lossy display-only transliteration of common Unicode punctuation to ASCII (smart quotes, dashes, ellipsis, arrows, symbols), with worst-case 4x expansion. | `/usr/src/usr.bin/mandoc/term_ascii.c` | **REFERENCE** | M | mandoc solves the same Unicode->ASCII fold problem but inside term_ascii.c (not a linkable API); mirror its table-driven approach, cannot link it | mandoc -Tascii does exactly this lossy fold but as program-internal code in usr.bin/mandoc/term_ascii.c — REFERENCE, not ADOPT. fugu's hand-maintained switch is fine; consider cross-checking its substitution table against mandoc's for coverage. No base library exports a transliteration function. | +| utf8_to_wcs | Decode a UTF-8 byte run into a freshly allocated wchar_t array using mbrtowc, substituting U+FFFD on invalid/incomplete sequences and resetting mbstate. | `/usr/src/usr.bin/sort/bwstring.c` mbrtowc(3) | **ADOPT** | S | already built on the base mbrtowc(3); error recovery should mirror base (reset state, advance 1 byte) which it does | **[verified correction: real mbrtowc caller (ssh utf8.c uses mbtowc, not mbrtowc)]** mbrtowc(3) is base libc (MBRTOWC(3), ). ssh/utf8.c:130-131 uses mbtowc + (void)mbtowc(NULL,...) to reset state on error; less/line.c:686 uses mbrtowc with mbstate. fugu's (size_t)-1/-2 handling + memset(&st) reset matches the idiom. Correct use of the base API. | +| render | Wrap every block to t->cols display columns (word-wrap on spaces, break too-wide glyphs) and, when drawing, emit visual lines with mvaddnwstr + attron/attroff; returns total visual line count. | `man curs_addwstr(3)` | **REFERENCE** | M | mvaddnwstr/attron are base-linkable and used correctly; the column-accurate word-wrap (lastsp tracking, too-wide-glyph break) is fugu-specific — align wrap edge cases to less/mandoc | **[verified correction: no base program demos mvaddnwstr; cite the man page]** mvaddnwstr is base wide curses (curs_addwstr(3), ncurses.h). less/line.c does the analogous wcwidth-driven column accounting for wrap (line.c:256,388,394). The wrapping algorithm itself is app layout (KEEP-ish) but the curses calls are base — net REFERENCE: keep the algorithm, ensure wcwidth==0 combining chars are handled as less does. | +| draw | Full-screen repaint: erase, render scrollback window, draw reverse-video status bar, draw the horizontally-scrolled input line tracking the cursor by display columns, position cursor, refresh. | curs_addstr(3) | **KEEP** | M | uses base curses primitives but the status-bar/input-line layout and horizontal-scroll-to-cursor are entirely app UI | All curses calls (erase/mvaddch/mvaddnstr/mvaddnwstr/move/clrtoeol/refresh/attron A_REVERSE) are base and used idiomatically, but the composed screen layout is fugu-specific rendering — nothing to adopt beyond the primitives already linked. | +| do_resize | On SIGWINCH: TIOCGWINSZ ioctl, resizeterm to the new size, re-read getmaxyx, force a full clear. | resizeterm(3) | **ADOPT** | S | resizeterm(3) and getmaxyx are base-linkable and already used; TIOCGWINSZ ioctl is the standard base resize idiom | resizeterm(3) is a base libcurses call (confirmed man resizeterm(3), ). The TIOCGWINSZ->resizeterm->getmaxyx->clearok sequence is the canonical curses resize handler. Already adopting the base API correctly. | +| drain_imsg | Read available imsg bytes then dispatch every decoded fugu_msg via handle_msg until the buffer drains. | imsg_init(3) | **KEEP** | S | thin pump over the project's ip_* imsg wrapper; the imsg layer is audited in imsgproto, not here | Underlying transport is base imsg (rewritten to imsgbuf_* in -current) but accessed through fugu's ip_read/ip_get abstraction — the adopt decision belongs to the imsgproto unit, not this UI file. Here it is pure dispatch glue. | +| handle_msg | Dispatch streamed protocol messages (M_DELTA/M_MESSAGE/M_TOOL_CALL/M_USAGE/M_TURN_DONE/M_SUMMARY/M_ERROR) into transcript blocks, convo state, journal appends, and status updates; parses tool-call/usage JSON with jsmn. | — | **KEEP** | M | application protocol state machine; JSON via vendored jsmn (allowed); no base equivalent | Pure fugu protocol semantics over convo/journal/json helpers. JSON parsing is the vendored jsmn permitted by charter. Nothing in base to align to. | +| handle_input | Translate a wint_t key (KEY_* function keys vs control/plain chars) into scrolling, cursor motion, line edits (backspace/delete/home/end/kill), quit, and submit. | curs_get_wch(3); libedit/editline(3) | **KEEP** | M | consumes base curses KEY_* codes but implements a bespoke single-line emacs-ish editor | **[verified correction: libedit IS in base (/usr/lib/libedit.so, man editline(3)) — the earlier "no editline in base" claim was wrong.]** Still KEEP, but for the right reason: libedit's `el_wgets` runs its own blocking read loop and cannot interleave with the curses `nodelay` get_wch + imsg `poll(2)` loop, which must edit the input line *while* streaming response deltas render. KEY_* come from get_wch with KEY_CODE_YES; the editor is correctly hand-rolled. | +| in_insert | Insert one wchar_t at the cursor, growing the wide input buffer (doubling) via xreallocarray and memmove-ing the tail. | reallocarray(3) | **KEEP** | S | app gap-free insert into a wchar_t buffer; only the allocator primitive is base and already wrapped | reallocarray(3) is base and already used via xreallocarray; the editor buffer management is app-specific. Nothing further to adopt. | +| submit | Encode the wide input line back to UTF-8 with wcrtomb, intercept slash-commands via ui_classify, else append a user block, update convo+journal, build the request and ip_compose/ip_flush an M_SUBMIT, set busy. | `/usr/src/lib/libc/citrus/citrus_utf8.c` wcrtomb(3) | **ADOPT** | S | the wcrtomb wide->multibyte encode is base libc and used correctly; the surrounding submit/convo/imsg flow is app logic (KEEP) but the encode primitive is a clean ADOPT | **[verified correction: wcrtomb impl (ssh utf8.c never calls wcrtomb)]** wcrtomb(3) is base libc (); fugu's MB_LEN_MAX buffer + mbstate_t + skip-on-(size_t)-1 loop is the standard inverse of the mbrtowc decode. No base program calls wcrtomb (only gnu/ subtrees), so cite the symmetric mbrtowc reference; the API itself is the win. Rest of submit (convo_build/ip_compose) is app KEEP. | +| do_command | Execute intercepted slash commands: /quit, /clear (journal marker + convo_clear), /compact (build convo, ip_compose M_SUMMARIZE, set compacting/busy). | — | **KEEP** | S | application command dispatch; no base equivalent | Pure fugu UX/protocol logic (clear/compact context management). Nothing in base to adopt or mirror. | + +### `src/ui/ui.c, src/ui/journal.c, src/ui/convo.c` + +_Mostly KEEP app-specific glue; two genuine base alignments: getline(3) is already correctly adopted for line input, and the hand-rolled write_all partial-write loop should align to base's atomicio (program-internal, REFERENCE) — plus open_memstream(3) is the base string-builder idiom behind fugu's local buf/json_writer._ + +| File:function | Purpose | Base equivalent (path:line / man) | Verdict | Eff | Risk / Tradeoff | Notes | +|---|---|---|---|---|---|---| +| line_replay | Format one replayed transcript part (user/tool/assistant/info) to stdout when resuming a session in line mode. | — | **KEEP** | S | app-specific transcript-formatting callback; no base equivalent | Plain printf with a fugu convo_part switch. Nothing to adopt. | +| ui_classify | Map a typed line to a slash-command enum (/clear, /compact, /quit, /exit). | — | **KEEP** | S | trivial strcmp command dispatch; no base option | A handful of strcmp(3) comparisons. Already idiomatic; nothing linkable to replace it. | +| line_collect | Pump the imsg channel to stdout for one reply turn (deltas, message persistence, tool calls, done/error). | — | **KEEP** | S | drives fugu imsgproto/convo/journal; no base option | Loop over ip_get/ip_read is fugu's own imsgproto wrapper (over base imsgbuf_*). The protocol pump itself is app logic. | +| line_compact | Wait for an M_SUMMARY reply to a /compact request, copying the summary into a buf. | — | **KEEP** | S | fugu imsgproto summary handshake; no base option | Same imsg pump pattern as line_collect, specialized for the summarize round-trip. | +| ui_line_run | The readline-free line-mode REPL: pledge/unveil narrowing, getline(3) read+chomp loop, command dispatch, submit/compact round-trips, journal append, failed-turn rollback. | `/usr/src/usr.bin/ftp/fetch.c:796` getdelim(3)/getline(3) | **KEEP** | S | orchestration glue that already uses the base getline(3) idiom correctly | This is the file's central line-input seam and it ALREADY adopts the base idiom: getline(&line,&lcap,stdin) with a manual \n/\r chomp and free(line) at exit — identical to base callers like ftp/fetch.c:796 and grep/file.c. Confirmed: declared /usr/include/stdio.h:159, man getdelim(3). No change needed; do NOT regress to fgets+fixed buffer. The surrounding submit/compact/journal flow is app logic (KEEP). | +| main | fugu-ui entry: log_init, ignore SIGPIPE, getopt(-a/-R), verify imsg fd 3 is a socket via fstat, isatty TTY detection, choose curses vs line front-end, teardown. | getopt(3) | **KEEP** | S | setup/dispatch glue already using base getopt/isatty/fstat idiomatically | Already uses the canonical base APIs (getopt while-switch, isatty+TERM probe, fstat+S_ISSOCK guard, signal(SIGPIPE,SIG_IGN)). Nothing to adopt or realign. | +| sessions_dir | Compose "$HOME/.fugu/sessions" into a caller buffer; fail if HOME unset or it overflows. | snprintf(3) | **KEEP** | S | getenv+snprintf path build; truncation-checked, already idiomatic | Correct snprintf >= bufsz truncation check. The base alternative for HOME expansion would be nothing more linkable; this is the right pattern. KEEP. | +| valid_id | Reject session ids that aren't a safe basename (no '/', not '.'/'..') before interpolating into a path. | — | **KEEP** | S | no base linkable path-component validator (base does this ad hoc per program) | Searched ssh/sftp-server.c and friends; base performs the same ../slash rejection inline per program (no shared API). fugu's strchr('/')+strcmp('.'/'..') is the standard hand-rolled check. KEEP. | +| write_all | Loop write(2) until all n bytes are written, retrying on EINTR; -1 on real error. | `/usr/src/usr.sbin/vmd/atomicio.c:56` | **REFERENCE** | S | base's atomicio() is the canonical full-write loop but is a per-program vendored file, not a linkable base symbol | Confirmed atomicio is NOT in /usr/include and NOT in libutil (util.h) — it is copied into ssh, nc, cvs, vmd, vmd, ldattach, sendbug independently. So we cannot link it (charter: base+jsmn only). fugu's write_all is correct but should mirror atomicio's edge cases at /usr/src/usr.sbin/vmd/atomicio.c:56-71: it handles EAGAIN (poll) and treats write returning 0 as EPIPE; write_all currently only handles EINTR. For a blocking journal fd that's fine, but align the 0-return/EAGAIN handling if the fd ever becomes non-blocking. REFERENCE: copy the idiom, don't link. | +| journal_unveil | unveil(2) the sessions dir rwc before the unveil lock so the journal fd can be created. | unveil(2) | **KEEP** | S | directly uses base unveil(2); nothing to adopt | Already a base syscall used idiomatically (best-effort: missing dir simply disables journaling). KEEP. | +| journal_open | Open/create the append-only jsonl session file: derive a sortable epoch-pid id (or validate a resume id), build the path, open O_WRONLY\|O_APPEND\|O_CREAT 0600. | `/usr/src/usr.bin/ftp/fetch.c` open(2) | **KEEP** | S | already uses the base O_APPEND\|O_CREAT open(2) idiom; id scheme is app-specific | O_APPEND open with 0600 perms matches base append-log callers (ftp/fetch.c, cvs/history.c use O_APPEND). Time-based id via time(NULL)+getpid avoids any timezone files (good for pledge). No linkable base helper for the id scheme. KEEP. | +| journal_foreach | Open a session jsonl by id and invoke a callback per non-empty, chomped line. | `/usr/src/usr.bin/ftp/fetch.c:874` getdelim(3)/getline(3) | **KEEP** | S | already uses the base fopen+getline(3) read-loop idiom correctly | This is the read-side line seam and it ALREADY adopts base: fopen("r") + while(getline(&line,&cap,fp)!=-1) with \n/\r chomp and free(line)+fclose — the exact pattern in ftp/fetch.c:874 and grep/file.c. Confirmed in /usr/include/stdio.h:159. No change; the surrounding callback dispatch is app logic. | +| journal_append | Append one json record plus a newline to the journal fd; disable journaling on first write error. | — | **KEEP** | S | thin wrapper over write_all (covered separately); no base option | Calls write_all twice (record + '\n'). The full-write idiom is the REFERENCE flagged on write_all; this wrapper itself is app policy (drop-on-error). KEEP. | +| journal_close | Close the journal fd if open and mark it -1. | close(2) | **KEEP** | S | trivial close(2) wrapper; no base option | Nothing to adopt. | +| convo_add_role | Append a {"role":..,"content":..} message built from plain text via fugu's json_writer into a buf, then xstrdup it into the message array. | `/usr/src/sbin/iked/ikev2.c:6508` open_memstream(3) | **KEEP** | M | replacing the project-wide buf/json_writer abstraction with open_memstream would not be a net win and crosses out of this file | FYI seam: open_memstream(3) IS base (declared /usr/include/stdio.h:256; real users iked/ikev2.c:6508, radiusd/radiusd_standard.c:345) and is the canonical base way to build a string by fprintf-ing into a FILE*. But fugu deliberately uses a vendored buf + json_writer shared project-wide; swapping in open_memstream here only (not the buf abstraction in buf.c) would be inconsistent. KEEP this function; consider open_memstream only if/when the buf abstraction itself is reviewed. | +| convo_add_user | Append a user-role message (thin wrapper over convo_add_role). | — | **KEEP** | S | one-line app wrapper; no base option | | +| convo_set_summary | Replace the active context with a /compact summary stored as a user/assistant pair (so role alternation the Messages API requires stays valid). | — | **KEEP** | S | app-specific Messages-API context model; no base option | Pure conversation-model policy. | +| convo_marker_clear | Build the journal marker {"fugu":"clear"} recording a context reset. | — | **KEEP** | S | app journal-marker schema via json_writer; no base option | Same json_writer/buf seam as convo_add_role; open_memstream note applies project-wide, not here. | +| convo_marker_compact | Build the journal marker {"fugu":"compact","summary":..} carrying the summary. | — | **KEEP** | S | app journal-marker schema; no base option | | +| convo_add_raw | Append a raw JSON message (copied, NUL-terminated) to the message array. | — | **KEEP** | S | xmalloc+memcpy+reallocarray growth of an app array; no base option | Uses fugu's x-allocators (over base reallocarray(3)); the array model is app logic. KEEP. | +| convo_truncate | Free messages from index n upward, shrinking the active context. | — | **KEEP** | S | app array teardown; no base option | | +| convo_free | Free all messages and the message array. | — | **KEEP** | S | app array teardown; no base option | | +| convo_build | Serialize the message array into a JSON array projection for each M_SUBMIT. | — | **KEEP** | S | concatenation into fugu's buf; no base option | Joins pre-built JSON message strings with commas inside [ ]. Pure app projection over buf. | +| replay_tool_use | Format one tool_use block (name + input) and hand it to the front-end like a live M_TOOL_CALL during replay. | — | **KEEP** | S | app replay formatting over jsmn json helpers; no base option | Uses fugu json_obj_get/json_tok_strdup (jsmn-backed). App logic. | +| convo_replay | Parse one stored message and emit display parts (string content, or array blocks: assistant text + tool_use; skip tool_result). | — | **KEEP** | M | app-specific Messages-API content-block walk over vendored jsmn; no base option | Charter-allowed jsmn is the JSON parser; the block-shape interpretation is fugu domain logic. KEEP. | +| record_kind | Classify a journal line as message / clear-marker / compact-marker (extracting summary) / skip. | — | **KEEP** | S | app journal record discriminator over jsmn; no base option | | +| resume_line | Apply one journal line during resume: keep+draw a message, reset context on /clear, or restore the summary on /compact — reconstructing the active context. | — | **KEEP** | M | app resume/replay state machine; no base option | Core of session resume semantics. App logic. | +| convo_resume | Replay a session journal by id back into a fresh convo (and via callback into the front-end transcript). | — | **KEEP** | S | thin driver over journal_foreach; no base option | Wires resume_ctx into journal_foreach (whose getline read-loop is the base idiom already in use). KEEP. | blob - /dev/null blob + 42bfacdc31a806121abd926fd0ee48884dc6edfc (mode 644) --- /dev/null +++ docs/base-idiom-audit.md @@ -0,0 +1,175 @@ +# fugu ↔ OpenBSD base: idiom audit brief + +**For:** a fresh agent/session with no prior context. +**Type:** read-only audit producing a written report. **Do not refactor in this pass.** + +## Goal + +fugu is an OpenBSD-native, privilege-separated AI coding agent in C (5 separate +binaries: `fugu` + `fugu-{ui,net,exec,fetch}`), sandboxed per-process with +`pledge(2)`/`unveil(2)`, base + a vendored `jsmn.h` only. While building it we +hand-rolled a number of utilities (a growable buffer, a JSON emitter, an +HTTP/1.1 client, an SSE parser, config-file secrecy checks, `poll(2)` event +loops, directory listing/sorting, etc.). + +**The task:** go function by function through fugu's source and decide, for each, +whether OpenBSD **base** already offers an idiomatic equivalent we should be +using or aligning to — a "seam" where our custom code overlaps something base +already does well. Produce a prioritized report. Implementation comes later, as a +separate pass, after the user reviews the findings. + +## Environment & access + +- **Host:** `ssh server` → OpenBSD 7.9-current, amd64. Non-interactive `ssh + server ''` works. **Never** `pkill -f fugu` on the host (it shares the ssh + session — use `pkill -x fugu-net` etc. if ever needed). +- **Base source:** `/usr/src` on the host (NOT `/var/src`). Ports: `/usr/ports`. + Headers also in `/usr/include`; manuals via `man`/`mandoc` on the host. +- **fugu source:** edit locally at `/home/isaac/org/tech/coding/fugu`, then + `rsync -a --exclude='*.o' / server:fugu/` and build on the host + (`ssh server 'cd fugu && make'`). The host copy is `~/fugu` (`/home/isaac/fugu`). + This audit is read-only, so you mostly just read the local tree and grep + `/usr/src` on the host. +- **Already validated:** the whole thing builds clean and passes 20 regress + suites; an OpenBSD port exists under `ports/fugu/`. None of that should change. + +## What counts as a "seam" — classify every function + +For each fugu function (or cohesive group), assign exactly one verdict: + +- **ADOPT** — base exposes a *linkable* idiomatic equivalent (a `libc`, + `libutil`, `libevent`, or `libtls` API, or a base header) that is a drop-in or + near-drop-in. These are the wins. Examples to confirm: `scandir(3)`, + `check_file_secrecy()`, `open_memstream(3)`, `recallocarray(3)`. +- **REFERENCE** — base *solves the same problem* but only inside a base program + (not a library we can link, e.g. `ftp(1)`'s HTTP client). We can't reuse the + code, but we should align our approach/idioms/edge-case handling to it. Cite + the base file and what to mirror. +- **KEEP** — no base equivalent exists; the custom code is justified. Record the + one-line reason (so the report is a complete accounting, not just a TODO list). + +A seam that would require a **non-base dependency** (a ports library) is out of +scope by fugu's charter — note it as KEEP with the reason "no base option." +`libevent` **is** base and therefore allowed, but see the tradeoff note below. + +## Method (how to check base — verify, don't assert) + +For every claim "base has X," prove it on the host: +1. **Man page:** `ssh server 'man 3 scandir'` (or section 1/2/5) — confirms it's a + supported public API, shows the signature. +2. **Header:** `grep -rn 'scandir' /usr/include` — confirms it's declared in base. +3. **A real user in base:** `grep -rln 'scandir(' /usr/src` — shows the idiomatic + call site to copy. Cite `path:line`. +Never claim an API from memory; OpenBSD prunes/renames (e.g. the imsg API was +rewritten to `imsgbuf_*`). If unsure whether something is a true library symbol +vs. a per-program static, check whether it's in a `lib/` dir or a program dir. + +## Source inventory (the work-list) + +29 source files under `fugu/src/` (ignore the generated `parent/parse.c`; audit +`common/parse.y`). Per-file purpose and the **lead** to investigate first: + +| fugu file | purpose | first lead in /usr/src | prior verdict | +|---|---|---|---| +| `common/log.c` | warn/warnx/fatal/log_* | `usr.sbin/bgpd/log.c`, `ntpd/log.c` (fugu's is derived from this; check for drift) | KEEP/SYNC | +| `common/util.c` | `xmalloc`/`xcalloc`/`xreallocarray`/`xstrdup`/`xasprintf` | `usr.bin/ssh/xmalloc.c` (canonical); libc `reallocarray`/`recallocarray`/`freezero` | REFERENCE | +| `common/buf.c` | growable byte buffer (`buf_init/append/printf/puts/putc/reserve/free`) | `lib/libutil/imsg-buffer.c` (`ibuf_*`); `open_memstream(3)` (user: `usr.sbin/radiusd/radiusd_standard.c`) | EVALUATE | +| `common/json.c` | JSON emitter + jsmn wrapper | none (no base JSON) | KEEP | +| `common/imsgproto.c` | chunked framing over libutil imsg | `lib/libutil/imsg.c` + `imsg-buffer.c` (`imsgbuf_*`, `imsg_compose`, `imsg_get_ibuf`); base has no chunking | REFERENCE/EVALUATE | +| `common/http.c` | HTTP/1.1 request build + chunked decode | `usr.bin/ftp/fetch.c` (base HTTP/HTTPS client) | REFERENCE | +| `common/sse.c` | server-sent-events parser | none | KEEP | +| `common/conf.c` | config model + `conf_check_secrecy` + merge/print | `check_file_secrecy()` in `usr.sbin/bgpd/parse.y`, `dvmrpd/parse.y`, `eigrpd/parse.y` | ADOPT | +| `common/confload.c` | `conf_load` (getpwuid/getgrouplist + StrictModes + overlay) | base daemons' config load; `getgrouplist(3)` | REFERENCE | +| `common/parse.y` | yacc grammar + lexer (`lgetc/lungetc/findeol/pushfile/popfile/symset/symget`) | `usr.sbin/httpd/parse.y` (the canonical template — diff against it) | SYNC (high value) | +| `common/url.c` | URL parse + RFC3986 encode + anti-injection | `usr.bin/ftp/fetch.c` (`url_get`/parsing) | REFERENCE | +| `common/html2text.c` | HTML→text | none | KEEP | +| `common/tlsconn.c` | libtls client config/connect/read/write + poll deadline | `usr.bin/ftp/fetch.c` libtls usage; `man tls_connect_socket`/`tls_config_set_protocols` | REFERENCE | +| `parent/parent.c` | privsep coordinator: spawn (fork+exec+socketpair), imsg relay (poll), secret delegation, session dir list/resolve (opendir+qsort), conf load | (a) `usr.sbin/httpd/proc.c` + `relayd/proc.c` (privsep process helper); (b) `sbin/dhcpleased/*.c` (libevent + imsg event loop); (c) `scandir(3)` (user: `usr.bin/diff/diffdir.c`) | EVALUATE (high value) | +| `net/net.c` | net worker main (tiny) | — | KEEP | +| `net/net_run.c` | Anthropic agent loop + imsg + http calls | request/response is synchronous — compare to base's blocking-client style | REFERENCE | +| `ui/ui.c` | line-mode front end + command dispatch | — | KEEP | +| `ui/ui_curses.c` | wide-char curses TUI | `lib/libcurses`; base curses users (`usr.bin/top`, `systat`) for wide-char/`get_wch` idioms | REFERENCE | +| `ui/journal.c` | append-only jsonl session log | none | KEEP | +| `ui/convo.c` | conversation model | none (app-specific) | KEEP | +| `exec/exec.c` | exec worker main + getopt + pledge | pledge/unveil idioms (already idiomatic) | KEEP | +| `exec/exec_run.c` | sandbox (unveil+execpromises) + tool dispatch | base privsep `unveil`/`pledge` + execpromises usage | KEEP/REFERENCE | +| `exec/tools_file.c` | read/write/edit (atomic temp+rename) | base atomic-write idiom (`mkstemp`+`rename`); `freezero` | REFERENCE | +| `exec/tools_shell.c` | `run_capture` (pipe+fork+exec+poll timeout) | base fork+exec+pipe capture (no `popen` — needs stderr+timeout) | KEEP/REFERENCE | +| `fetch/fetch.c` | fetch worker main | — | KEEP | +| `fetch/fetch_run.c` | SSRF guard (getaddrinfo + addr_blocked) + libtls fetch | `getaddrinfo(3)` idioms; no base SSRF helper | KEEP/REFERENCE | +| `fetch/kagi.c` | Kagi API request/format | none | KEEP | + +## High-value seams (confirmed; investigate these first) + +These four are the most likely real wins — all confirmed present in base: + +1. **`conf_check_secrecy` → base `check_file_secrecy()`** (`bgpd/parse.y` et al.). + The base function (StrictModes-style 0600/owner check) is copied verbatim across + daemons. fugu should match it byte-for-byte so reviewers recognize it. +2. **`parent` directory listing → `scandir(3)` + comparator** (`diff/diffdir.c`). + `resolve_last()`/`list_sessions()` hand-roll opendir+readdir+`qsort`; `scandir` + is the one-call idiom with a sort comparator. +3. **`common/parse.y` lexer → diff against `usr.sbin/httpd/parse.y`.** Our lexer is + the classic pf/httpd template; base has since evolved it (e.g. `igetc`, + `expand_macros`, growable token buffer, `check_file_secrecy`). Sync to current. +4. **`buf.c` → `ibuf`/`open_memstream`.** Decide deliberately: `ibuf` (libutil) for + wire/byte buffers, `open_memstream(3)` for string building. Tradeoff: `buf` is + tiny and dependency-free; quantify what adopting either actually buys. + +## The big design question (flag, don't pre-decide) + +**`poll(2)` loops → `libevent` + the base imsg-privsep pattern.** Every base +privsep daemon (`dhcpleased`, `slaacd`, `unwind`, `httpd`, `relayd`) is built on +`libevent` (`event_init`/`event_add`/`event_dispatch`) plus a shared +`imsg_event_add`/`dispatch_imsg`/`proc.c` idiom. fugu hand-rolls `poll()` in the +parent relay and each worker. This is the single largest stylistic divergence +from base — **but** fugu is deliberately foreground, per-invocation, and *not* a +daemon, and its flows are largely synchronous request/response. Treat this as an +architectural recommendation with an honest tradeoff analysis (idiomatic-ness and +`proc.c` reuse vs. added machinery for a non-daemon), **not** an assumed adoption. +Likewise evaluate whether `parent.c`'s spawn/relay should be rebuilt on +`httpd/proc.c`'s privsep helper. + +## Output: the deliverable + +Write findings to `fugu/docs/base-idiom-audit-findings.md`: + +- **Top of file:** a prioritized "Adopt these" list (the clear ADOPT wins), then + "Evaluate" (tradeoff calls), then "Keep (justified)" — each one line. +- **Body:** one row per fugu function (or tight group): + `File:function | Purpose | Base equivalent (path:line + man page) | Verdict | Effort (S/M/L) | Risk/Tradeoff | Notes`. +- Every ADOPT/REFERENCE row must cite a real `path:line` in `/usr/src` (or a man + page) you actually looked at. KEEP rows state the reason in one phrase. +- It must be a *complete* accounting: every function appears exactly once. + +## Suggested execution (this is a parallelizable, read-heavy audit) + +Run it as a multi-agent **Workflow** (the user has opted into orchestration for +this; confirm "ultracode"/explicit opt-in is still in effect before launching): + +- **Phase 1 — audit (fan out):** one agent per source file or tight group (~14 + units; e.g. group `net/*`, `exec/*`, `fetch/*`, `ui/*`, and each `common/*` + individually since common holds the densest seams). Each agent: reads the fugu + file locally, enumerates its functions, then for each greps `/usr/src` + + checks the man page/header on `server`, and returns **structured** findings + (use a JSON schema: array of `{func, purpose, base_equiv, path_line, verdict, + effort, tradeoff}`). Give every agent this brief's "Method" and "classify" + rules and the relevant row(s) from the inventory as its starting lead. +- **Phase 2 — synthesize (single agent):** merge all findings, de-dup + cross-cutting ones (e.g. `check_file_secrecy` touches conf.c + parse.y), sort + into the Adopt/Evaluate/Keep buckets, and write the findings file. Then a + **completeness check**: assert every inventory file is covered and every + ADOPT/REFERENCE claim carries a real citation; re-audit anything missing. + +## Guardrails / non-goals + +- **Read-only.** Produce the report; change no code, Makefiles, or the port. +- **Honor fugu's charter:** OpenBSD-only, base + vendored `jsmn` only, per-process + `pledge`/`unveil`, secrets never in argv/env/journal. Reject any seam needing a + ports dependency. +- **Verify every base claim** against the actual tree/man page with a citation; + the imsg API in particular was rewritten — confirm current symbols. +- **Separate library reuse (ADOPT) from style alignment (REFERENCE).** Don't + recommend linking a base *program's* internal code. +- For design-level seams (libevent, `proc.c`), present the tradeoff and a + recommendation; the user decides. blob - /dev/null blob + 93124eb9cb6a5e0b43ff1618811cc1b0577b980a (mode 644) --- /dev/null +++ etc/fugu.conf.sample @@ -0,0 +1,69 @@ +# fugu.conf.sample - configuration for fugu(1). +# +# Installed as /etc/fugu.conf.sample. Copy to /etc/fugu.conf to use. fugu reads +# ALL of its configuration -- including the secrets (api_key, anthropic_key, +# kagi_token) -- from this single file. It is installed set-group-id _fugu, so +# the parent can read a root-owned /etc/fugu.conf that the model's tools cannot: +# +# install -o root -g _fugu -m 0640 /etc/fugu.conf.sample /etc/fugu.conf +# +# It MUST be mode 0640 (or stricter), owned by root, group _fugu; fugu refuses to +# start if it is world-accessible or group-writable. See fugu.conf(5). + +# Default model and per-turn generation budget. +#model "claude-sonnet-4-6" +#max_tokens 4096 + +# API wire protocol: "anthropic" (default) or "openai". The "openai" provider +# speaks chat/completions, which most other vendors emulate; point api_host at +# the vendor, override api_path if it does not serve at /v1, and set a matching +# model and api_key. +#provider "openai" +#api_host "openrouter.ai" +#api_path "/api/v1/chat/completions" # OpenRouter; default is /v1/chat/completions +#model "anthropic/claude-sonnet-4.6" + +# Several providers at once. The flat keys above define the default provider; +# each "provider" block adds another, switchable at runtime with /model (which +# lists every provider's models, tagged by name). See fugu.conf(5). +#provider "gpt" { +# type "openai" +# api_key_file "/home/me/.fugu/openai.key" +# model "gpt-5" +#} + +# System prompt sent on every turn. Unset = fugu's concise built-in default +# (identity, terse-tone rules, scope/style defaults, tool guidance); set this +# to replace it. A string setting, so keep it on one logical line; end a line +# with \ to continue. +#system "You are a terse assistant for an OpenBSD C codebase." + +# Advisory token budget for the conversation sent to the model. +#context_limit 200000 + +# Let the file tools modify the working tree (default yes). +#allow_write yes + +# Let commands run by the shell tool use the network (default no). When no, +# the exec process and its children are pledged without "inet", so commands +# such as ftp(1), nc(1) and "git fetch" fail by design. +#allow_subprocess_net no + +# Offer the brokered web search/fetch tools to the model. +#web_search yes + +# Let the model make arbitrary HTTPS requests (the http_request tool) to a +# restricted set of hosts. Only offered when http_allow is set; a request must +# match an http_allow pattern and no http_block pattern. Patterns are +# host[/pathprefix]; a leading dot matches subdomains. fugu adds no credentials +# of its own -- the model supplies its own headers. See fugu.conf(5). +#http_allow "api.github.com .githubusercontent.com" +#http_block "api.github.com/admin" + +# Render the interface as ASCII instead of UTF-8 (default no). +#ascii no + +# Per-user / per-group scoping, first match wins (see fugu.conf(5)). +#match group "wheel" { +# allow_subprocess_net yes +#} blob - /dev/null blob + b4ad8ceb3a3eb594b00d0ab490ecc51d3b4cfc82 (mode 644) --- /dev/null +++ man/fugu.1 @@ -0,0 +1,316 @@ +.\" Copyright (c) 2026 Isaac +.\" +.\" Permission to use, copy, modify, and distribute this software for any +.\" purpose with or without fee is hereby granted, provided that the above +.\" copyright notice and this permission notice appear in all copies. +.\" +.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +.\" +.Dd June 21, 2026 +.Dt FUGU 1 +.Os +.Sh NAME +.Nm fugu +.Nd OpenBSD-native AI coding agent +.Sh SYNOPSIS +.Nm +.Op Fl cdlnv +.Op Fl r Ar id +.Sh DESCRIPTION +.Nm +is an interactive AI coding agent that pairs the Anthropic Messages API with a +set of local tools \(en reading, writing and editing files, running +.Xr ksh 1 +commands, searching the tree, and brokered web search and fetch \(en so that a +language model can work inside a project under the user's direction. +.Pp +.Nm +is built as five privilege-separated programs that the parent +.Xr fork 2 Ns / Ns Xr execve 2 Ns s +at startup and that speak to one another only through the parent over +.Xr imsg 3 : +.Pp +.Bl -tag -width fugu-fetchXX -compact +.It Nm +parent: reads configuration, delegates secrets, and relays every +worker-to-worker message. +Pledge +.Qq stdio proc . +.It Nm fugu-ui +the terminal front end (a +.Xr curses 3 +transcript on a tty, or a line reader otherwise). +.It Nm fugu-net +the only process holding the model API key (or keys, when several providers are +configured); it speaks TLS to the configured model endpoints and nothing else. +.It Nm fugu-exec +runs the file and shell tools with +.Em no +network access; subprocesses are confined to the project directory. +.It Nm fugu-fetch +performs web search and fetch and parses untrusted HTML in an address space +that holds neither the API key nor the ability to +.Xr execve 2 . +.El +.Pp +Each process restricts itself with +.Xr pledge 2 +and +.Xr unveil 2 +to the least authority it needs, so a flaw in one (a prompt injection that +subverts tool execution, a memory bug in HTML parsing) cannot reach the +network, the credential, or files outside the working tree. +.Nm +holds no +.Xr setuid 2 +privilege and runs no daemon, one invocation per session. +Its only privilege is set-group-id +.Sy _fugu , +used by the parent solely to read the root-owned +.Pa /etc/fugu.conf ; +the parent sheds that group as soon as the secrets are delegated, and every +worker sheds it at startup, so neither the workers nor the shell tools they run +can reach +.Pa /etc/fugu.conf +or any other +.Sy _fugu Ns -readable +file. +.Pp +On startup +.Nm +reads its configuration (see +.Xr fugu.conf 5 ) , +opens a session journal under +.Pa ~/.fugu/sessions , +and presents a prompt. +Type a message and press +.Ic Enter +to send it; the assistant's reply streams back, and any tool calls it makes are +shown as they run. +.Pp +The options are as follows: +.Bl -tag -width Ds +.It Fl c +Resume the most recently modified session, continuing its journal. +.It Fl d +Keep logging to standard error for the whole run, including while the session is +active. +Startup errors \(en such as a configuration syntax error \(en always go to +standard error; once the interface takes over the terminal, logging normally +switches to +.Xr syslog 3 +so it does not paint over the display. +.Fl d +suppresses that switch, which is useful for debugging at the cost of a garbled +screen. +.It Fl l +List the saved sessions \(en id, last-modified time, message count and a +preview of the first message \(en newest first, then exit. +.It Fl n +Parse the configuration, resolve the effective settings for the invoking user, +print them with secrets redacted, and exit without starting a session. +.It Fl r Ar id +Resume the session with the given +.Ar id +(as shown by +.Fl l ) , +continuing its journal. +.It Fl v +Be more verbose; repeat for more detail. +.El +.Sh COMMANDS +While a session is running, a line beginning with a slash is interpreted as a +command rather than sent to the model: +.Bl -tag -width "/compactXX" +.It Ic /clear +Reset the active context to empty so the next message starts a fresh +conversation. +The journal is preserved on disk and the prior transcript stays on screen as +scrollback; a marker records the reset. +.It Ic /compact +Replace the active context with a model-written summary of the conversation so +far (one extra API call), reducing the tokens carried into later turns. +The full journal is preserved; a marker records the summary. +.It Ic /model +Open a fuzzy-searchable menu of the models offered by every configured provider +and switch to the one chosen for the rest of the session. +The list is fetched live from each provider's model endpoint; when more than one +provider is configured each entry is tagged with its provider name, and choosing +a model also makes its provider the active one +.Pq see Sx PROVIDER BLOCKS in Xr fugu.conf 5 . +In the menu, type to filter, +.Ic Up Ns / Ns Ic Down +(or +.Ic ^P Ns / Ns Ic ^N ) +to move the selection, +.Ic Enter +to choose, and +.Ic Esc +to cancel. +.It Ic /quit , Ic /exit +End the session. +.El +.Pp +A slash name that is not one of the built-ins above is looked up in +.Pa ~/.fugu/skills/ +.Pq see Sx FILES : +on a hit, the skill's body is submitted as a user message, with any text typed +after the name appended as a second paragraph. +A marker records the invocation in the journal so a resumed session reproduces +it. +.Pp +.Ic Tab +.Pq or Ic Shift-Tab No to go backward +on a partial slash command cycles through the commands that match what has been +typed, built-ins first and then loaded skills alphabetically. +.Sh KEYS +The +.Xr curses 3 +front end edits the input line with modal, +.Xr vi 1 Ns -like +keys. +It starts in insert mode, where typed characters are inserted at the cursor; +.Ic Esc +switches to normal mode for navigation and editing commands. +.Pp +These keys work in either mode: +.Bl -tag -width "PgUp / PgDnXX" +.It Ic Enter +Send the current line. +.It Ic Tab , Ic Shift-Tab +On a line beginning with a slash, cycle forward +.Pq Ic Tab +or backward +.Pq Ic Shift-Tab +through the slash commands that match what has been typed. +.It Ic PgUp / PgDn +Scroll the transcript a page at a time; the newest output is shown again after +.Ic PgDn . +.It Ic Up / Down +Recall the previous or next prompt sent this session. +.It Ic Ctrl-L +Repaint the screen. +.It Ic Ctrl-C +Clear the input line, or quit when it is already empty. +.It Ic Ctrl-D +Quit when the input line is empty. +.El +.Pp +In normal mode: +.Bl -tag -width "i a A I o OXX" +.It Ic h l 0 $ ^ w b e +Move by character, to the start or end of the line, to the first non-blank, or +by word. +.It Ic i a A I o O +Enter insert mode \(en before or after the cursor, or at the start or end of +the line. +.It Ic x D dd +Delete the character under the cursor, to end of line, or the whole line; +.Ic d +also takes a motion +.Pq Ic dw , de , db , d$ , d0 . +.It Ic j / k +Recall the next or previous prompt sent this session. +.It Ic J / K +Scroll the transcript down or up one line. +.El +.Sh ENVIRONMENT +.Bl -tag -width FUGU_CONF +.It Ev HOME +Location of the session directory +.Pa ~/.fugu/sessions . +.It Ev FUGU_CONF +Alternate path to the configuration file, for development and the test suite. +Honoured only when +.Nm +is +.Em not +installed set-group-id +.Pq Xr issetugid 2 ; +an installed +.Nm +ignores it and always reads +.Pa /etc/fugu.conf , +so it cannot be steered onto another +.Sy _fugu Ns -readable +file. +.El +.Sh FILES +.Bl -tag -width "~/.fugu/sessions/XX" +.It Pa /etc/fugu.conf +Configuration and secrets; mode +.Ar 0640 , +owner +.Sy root , +group +.Sy _fugu +(see +.Xr fugu.conf 5 ) . +.It Pa ~/.fugu/sessions/ +Append-only session journals, one +.Pa .jsonl +per session. +.It Pa ~/.fugu/skills/ +Slash-invoked prompt templates. +Each +.Pa /SKILL.md +is a markdown file with an optional +.Sq --- +fenced header carrying +.Sq name +and +.Sq description +lines; the body below the closing fence is what the model sees when the user +types +.Sy / Ns Ar name . +Loaded once at startup; create or edit files between sessions. +.El +.Sh EXIT STATUS +.Ex -std +.Sh SEE ALSO +.Xr ksh 1 , +.Xr pledge 2 , +.Xr unveil 2 , +.Xr imsg 3 , +.Xr fugu.conf 5 +.Sh STANDARDS +.Nm +speaks the Anthropic Messages API with streaming server-sent events. +.Sh HISTORY +.Nm +is a clean-room reimplementation, in behavior rather than source, of the +TypeScript agent +.Em pi +by Mario Zechner. +.Sh AUTHORS +.An Isaac Aq Mt isaac@itm.works . +.Sh CAVEATS +.Nm +runs tools and shell commands chosen by a language model. +Although +.Xr pledge 2 +and +.Xr unveil 2 +confine those tools to the working tree and deny them the network, the model +can still modify, delete, or read any file beneath the directory in which +.Nm +is started. +Run it from the project you intend it to change, and review what it does. +In particular, when started from within your home directory the tools can read +.Pa ~/.fugu/sessions , +so the model may see other sessions' transcripts; the configuration secrets, +however, are not reachable +.Pq they live in the Sy _fugu Ns -only Pa /etc/fugu.conf . +.Pp +By default subprocesses started by the shell tool have no network access; +set +.Ic allow_subprocess_net +in +.Xr fugu.conf 5 +to permit it. blob - /dev/null blob + eddfbc12ede46945df2f202c02619486e18a086d (mode 644) --- /dev/null +++ man/fugu.conf.5 @@ -0,0 +1,434 @@ +.\" Copyright (c) 2026 Isaac +.\" +.\" Permission to use, copy, modify, and distribute this software for any +.\" purpose with or without fee is hereby granted, provided that the above +.\" copyright notice and this permission notice appear in all copies. +.\" +.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +.\" +.Dd June 21, 2026 +.Dt FUGU.CONF 5 +.Os +.Sh NAME +.Nm fugu.conf +.Nd configuration file for fugu +.Sh DESCRIPTION +.Xr fugu 1 +reads all of its configuration, including the secrets, from a single file: +.Pp +.Bl -tag -width "/etc/fugu.confXX" -compact +.It Pa /etc/fugu.conf +Owned by +.Sy root , +group +.Sy _fugu , +mode +.Ar 0640 . +.El +.Pp +Because +.Xr fugu 1 +is installed set-group-id +.Sy _fugu , +the parent can read this root-owned file while the model's file and shell tools, +which run as the invoking user without that group, cannot +.Pq see Sx FILES and the Sx CAVEATS in Xr fugu 1 . +Secrets +.Po +.Ic api_key , +.Ic anthropic_key , +.Ic kagi_token +.Pc +therefore live here too, administered centrally rather than in each user's home. +.Pp +Effective settings are resolved low to high precedence: compiled defaults, the +global section, then the first matching +.Ic match +block. +The merged result for the invoking user can be printed with +.Ic fugu -n , +which redacts secret values. +.Pp +The file holds secrets, so the parent refuses to start if it is +world-accessible or group-writable, or owned by anyone but +.Sy root +or the invoking user. +For development and the test suite, a binary that is +.Em not +set-group-id honours the +.Ev FUGU_CONF +environment variable as the path to read instead +.Pq see Xr fugu 1 ; +an installed, set-group-id +.Xr fugu 1 +ignores the environment so it cannot be steered onto another +.Sy _fugu Ns -readable +file. +.Sh SYNTAX +The grammar follows the +.Xr pf.conf 5 Ns / Ns Xr httpd.conf 5 +family. +Comments begin with +.Sq # . +Argument strings are enclosed in double quotes. +A boolean is the word +.Ic yes +or +.Ic no . +Macros may be defined and later expanded with a leading +.Sq $ : +.Bd -literal -offset indent +key = "value" +anthropic_key $key +.Ed +.Pp +Other files may be pulled in with +.Ic include Ar path . +.Sh SETTINGS +The following keywords may appear in the global section or inside a +.Ic match +block: +.Bl -tag -width Ds +.It Ic model Ar string +The model id to use, as named by the configured +.Ic provider . +Defaults to +.Qq claude-sonnet-4-6 , +the default Anthropic model. +.It Ic provider Ar string +The API wire protocol to speak, one of +.Qq anthropic +.Pq the default +or +.Qq openai . +The +.Qq openai +provider speaks the OpenAI +.Pa /v1/chat/completions +API that most other vendors and local servers emulate +.Pq OpenRouter , Groq , Together , DeepSeek , and Ollama , llama.cpp , vLLM , +authenticating with +.Ic api_key +as a +.Ar Bearer +token. +Set +.Ic api_host +to the vendor's host +.Pq and usually a +.Ic model +and +.Ic api_key +to match . +.It Ic system Ar string +System prompt sent to the model on every turn, used to set the assistant's +role, conventions, or constraints. +Unset by default, in which case fugu sends a concise built-in default prompt; +set this to replace that default entirely. +The value is a single quoted string +.Pq as with any string setting, an embedded literal newline is not preserved , +so write a multi-sentence prompt as one line. +.It Ic max_tokens Ar number +Maximum number of tokens the model may generate per turn. +.It Ic context_limit Ar number +Advisory token budget for the conversation projection sent to the model. +.It Ic allow_write Ar yes Ns | Ns Ar no +Whether the file tools may modify the working tree. +Default +.Ic yes . +.It Ic allow_subprocess_net Ar yes Ns | Ns Ar no +Whether commands run by the shell tool may use the network. +Default +.Ic no : +the exec process and its children are pledged without +.Ar inet , +so network-using commands +.Po Xr ftp 1 , Xr nc 1 , git fetch Pc +fail by design until this is enabled. +.It Ic web_search Ar yes Ns | Ns Ar no +Whether the brokered +.Ic web_search +and +.Ic web_fetch +tools are offered to the model. +Default +.Ic yes ; +set +.Ic no +to withhold both tools, so the model cannot reach the network at all. +.It Ic http_allow Ar string +A whitespace-separated list of host patterns the model may reach with the +general +.Ic http_request +tool, which makes an arbitrary-method HTTPS request with model-supplied headers +and body. +The tool is offered to the model +.Sy only +when this list is set; with no +.Ic http_allow +it does not exist. +A request is permitted when its host matches at least one +.Ic http_allow +pattern and no +.Ic http_block +pattern. +Each pattern is +.Ar host Ns Op / Ns Ar pathprefix : +.Bl -bullet -compact -offset indent +.It +.Qq example.com +matches that host exactly; +.It +.Qq .example.com +matches +.Qq example.com +and any subdomain of it; +.It +an optional +.Pa / Ns Ar pathprefix +further requires the request path to begin with it +.Pq Qq api.example.com/v1 . +.El +Host matching is case-insensitive. +fugu injects no credentials of its own, so the model can only reach hosts the +operator already lists, using whatever headers it supplies; the +.Ic web_fetch +SSRF guard +.Pq private, loopback and link-local addresses refused +also applies. +.It Ic http_block Ar string +A whitespace-separated list of patterns, in the same form as +.Ic http_allow , +that are denied even when an +.Ic http_allow +pattern would otherwise permit them +.Pq a block wins . +.It Ic search_provider Ar string +Name of the web search provider. +.It Ic ascii Ar yes Ns | Ns Ar no +Render the terminal interface transliterated to ASCII instead of UTF-8. +Default +.Ic no . +.It Ic api_host Ar string +Hostname of the model API endpoint, overriding the provider's built-in default +.Po +.Qq api.anthropic.com +or +.Qq api.openai.com +.Pc . +Required for third-party +.Qq openai +endpoints +.Pq e.g. Qq openrouter.ai . +.It Ic api_port Ar string +TCP port of the model API endpoint. +Defaults to +.Qq 443 . +fugu connects over TLS only, so the endpoint must speak HTTPS. +.It Ic api_path Ar string +Request path of the chat/completions endpoint, overriding the provider's +default +.Po +.Qq /v1/messages +or +.Qq /v1/chat/completions +.Pc . +Needed for OpenAI-compatible vendors that do not serve at +.Qq /v1 , +for example +.Qq /api/v1/chat/completions +.Pq OpenRouter +or +.Qq /openai/v1/chat/completions +.Pq Groq . +.It Ic api_key Ar string +The model API key, used for any provider. +.Sy Secret ; +kept in the root-owned +.Pa /etc/fugu.conf . +For the +.Qq anthropic +provider the legacy +.Ic anthropic_key +is an accepted synonym; +.Ic api_key +takes precedence when both are set. +.It Ic api_key_file Ar string +Path to a file whose first line is the model API key, used instead of +.Ic api_key . +.It Ic anthropic_key Ar string +The Anthropic API key; a legacy synonym for +.Ic api_key +kept for compatibility. +.Sy Secret ; +kept in the root-owned +.Pa /etc/fugu.conf . +.It Ic anthropic_key_file Ar string +Path to a file whose first line is the Anthropic API key, used instead of +.Ic anthropic_key . +.It Ic kagi_token Ar string +The Kagi API token used by the web tools. +.Sy Secret ; +kept in the root-owned +.Pa /etc/fugu.conf . +.It Ic kagi_token_file Ar string +Path to a file whose first line is the Kagi token, used instead of +.Ic kagi_token . +.El +.Sh MATCH BLOCKS +.Pa /etc/fugu.conf +may scope settings to particular users or groups with first-match-wins +.Ic match +blocks, in the style of +.Xr ssh_config 5 Ns 's +.Ic Match : +.Bd -literal -offset indent +match user "alice" { + model "claude-opus-4-8" + allow_subprocess_net yes +} + +match group "students" { + allow_write no +} +.Ed +.Pp +The first block whose +.Ic user +or +.Ic group +matches the invoking user supplies the scoped settings; later blocks are not +consulted. +Secrets may not appear in a +.Ic match +block, as they may not appear anywhere in +.Pa /etc/fugu.conf . +.Sh PROVIDER BLOCKS +The flat +.Ic provider , model , api_key , api_host , api_port +and +.Ic api_path +settings above configure a single, default provider. +To make more than one provider available at once \(en a direct Anthropic key +.Em and +an OpenAI-compatible endpoint, say \(en give each additional one a named +.Ic provider +block: +.Bd -literal -offset indent +provider "name" { + type "anthropic" | "openai" + model string + api_key string + api_key_file string + api_host string + api_port string + api_path string +} +.Ed +.Pp +.Ic type +selects the wire protocol +.Pq it is the in-block spelling of the flat Ic provider No keyword ; +the remaining keys mean exactly what they do at the top level, but scoped to +this provider. +The flat top-level keys still define an implicit +.Qq default +provider, so existing single-provider configurations are unchanged; each +.Ic provider +block adds one more. +At startup the default provider (or, if none is configured, the first block) is +active; the +.Ic /model +command +.Pq see Xr fugu 1 +lists the models of every configured provider, tagged by name, and switching to +one also switches the active provider for the rest of the session. +A block's +.Ic api_key +is a secret like any other and lives in the same root-owned +.Pa /etc/fugu.conf ; +.Ic api_key_file +may point at a separate file the +.Sy _fugu +group can read instead. +.Sh FILES +.Bl -tag -width "/etc/fugu.conf.sampleXX" -compact +.It Pa /etc/fugu.conf +The configuration and secrets; mode +.Ar 0640 , +owner +.Sy root , +group +.Sy _fugu . +.It Pa /etc/fugu.conf.sample +Annotated example installed with the package. +.El +.Sh EXAMPLES +A minimal +.Pa /etc/fugu.conf +.Pq Sy root : Ns Sy _fugu No mode Ar 0640 : +.Bd -literal -offset indent +anthropic_key "sk-ant-..." +model "claude-sonnet-4-6" +system "You are a terse assistant for an OpenBSD C codebase. \e +Follow KNF, prefer base over ports, and explain risky changes." +.Ed +.Pp +Reading the key from a separate file, and enabling the web tools: +.Bd -literal -offset indent +anthropic_key_file "/etc/fugu/anthropic.key" +kagi_token_file "/etc/fugu/kagi.token" +web_search yes +.Ed +.Pp +Using an OpenAI-compatible provider +.Pq here OpenRouter , whose endpoint is not at Qq /v1 : +.Bd -literal -offset indent +provider "openai" +api_host "openrouter.ai" +api_path "/api/v1/chat/completions" +model "anthropic/claude-sonnet-4.6" +api_key "sk-or-..." +.Ed +.Pp +Groq's free tier +.Pq endpoint at Qq /openai/v1 : +.Bd -literal -offset indent +provider "openai" +api_host "api.groq.com" +api_path "/openai/v1/chat/completions" +model "llama-3.3-70b-versatile" +api_key "gsk-..." +.Ed +.Pp +Two providers at once: a direct Anthropic key as the default, plus a named +OpenAI provider. +.Ic /model +then lists and switches between both. +.Bd -literal -offset indent +anthropic_key "sk-ant-..." +model "claude-sonnet-4-6" + +provider "gpt" { + type "openai" + api_key_file "/etc/fugu/openai.key" + model "gpt-5" +} +.Ed +.Sh SEE ALSO +.Xr fugu 1 +.Sh CAVEATS +Because +.Pa /etc/fugu.conf +holds secrets, a file that is world-accessible, group-writable, or owned by +neither +.Sy root +nor the invoking user is a fatal error rather than a warning: +.Xr fugu 1 +refuses to start until it is corrected. blob - /dev/null blob + 5afa5d62906884cb805efc418be4b2e7d2d8094f (mode 644) --- /dev/null +++ regress/anthropic/Makefile @@ -0,0 +1,19 @@ +# Unit tests for the Anthropic provider layer (src/common/anthropic.c). + +PROG= anthropic_test +SRCS= anthropic_test.c anthropic.c sse.c json.c buf.c util.c log.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +LDADD+= -lutil +DPADD+= ${LIBUTIL} + +REGRESS_TARGETS= run-anthropic_test + +run-anthropic_test: ${PROG} + ./${PROG} + +.include blob - /dev/null blob + d0a75de6bd444fcf0a156298a3b1a10b2c20737d (mode 644) --- /dev/null +++ regress/anthropic/anthropic_test.c @@ -0,0 +1,428 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include +#include + +#include "anthropic.h" +#include "buf.h" +#include "sse.h" + +static int checks; +static int failures; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +/* ---- callback recorder ---- */ + +static struct buf rec_text; +static char *rec_tool_id, *rec_tool_name, *rec_tool_input; +static int rec_tools; +static long long rec_in, rec_out; +static int rec_usage; +static char *rec_stop; +static int rec_done; +static char *rec_err_type, *rec_err_msg; +static int rec_errors; + +static void +rec_reset(void) +{ + buf_reset(&rec_text); + free(rec_tool_id); rec_tool_id = NULL; + free(rec_tool_name); rec_tool_name = NULL; + free(rec_tool_input); rec_tool_input = NULL; + rec_tools = 0; + rec_in = rec_out = 0; + rec_usage = 0; + free(rec_stop); rec_stop = NULL; + rec_done = 0; + free(rec_err_type); rec_err_type = NULL; + free(rec_err_msg); rec_err_msg = NULL; + rec_errors = 0; +} + +static void +cb_text(const char *t, size_t n, void *a) +{ + (void)a; + buf_append(&rec_text, t, n); +} + +static void +cb_tool(const char *id, const char *name, const char *input, size_t n, void *a) +{ + (void)a; + free(rec_tool_id); rec_tool_id = strdup(id); + free(rec_tool_name); rec_tool_name = strdup(name); + free(rec_tool_input); + rec_tool_input = malloc(n + 1); + memcpy(rec_tool_input, input, n); + rec_tool_input[n] = '\0'; + rec_tools++; +} + +static void +cb_usage(long long in, long long out, void *a) +{ + (void)a; + rec_in = in; + rec_out = out; + rec_usage++; +} + +static void +cb_stop(const char *sr, void *a) +{ + (void)a; + free(rec_stop); + rec_stop = strdup(sr); +} + +static void +cb_error(const char *type, const char *msg, void *a) +{ + (void)a; + free(rec_err_type); rec_err_type = strdup(type); + free(rec_err_msg); rec_err_msg = strdup(msg); + rec_errors++; +} + +static void +cb_done(void *a) +{ + (void)a; + rec_done++; +} + +static const struct llm_cbs RECORDER = { + cb_text, cb_tool, cb_usage, cb_stop, cb_error, cb_done +}; + +static void +ev(struct anthropic_stream *s, const char *event, const char *data) +{ + anthropic_stream_event(s, event, data, strlen(data)); +} + +/* ---- request builder ---- */ + +static void +test_build_basic(void) +{ + struct buf out; + struct anth_msg msgs[] = { + { "user", "hello" }, + { "assistant", "hi there" }, + }; + const char *expect = + "{\"model\":\"claude-sonnet-4-6\",\"max_tokens\":1024," + "\"stream\":true,\"messages\":[{\"role\":\"user\"," + "\"content\":\"hello\"},{\"role\":\"assistant\"," + "\"content\":\"hi there\"}]}"; + + buf_init(&out); + anthropic_build_request(&out, "claude-sonnet-4-6", 1024, NULL, + msgs, 2, NULL); + buf_terminate(&out); + CHECK(strcmp(out.data, expect) == 0, "build basic:\n%s", out.data); + buf_free(&out); +} + +static void +test_build_system_tools(void) +{ + struct buf out; + struct anth_msg msgs[] = { { "user", "hi" } }; + const char *expect = + "{\"model\":\"m\",\"max_tokens\":8,\"stream\":true," + "\"system\":\"be brief\",\"messages\":[{\"role\":\"user\"," + "\"content\":\"hi\"}],\"tools\":[{\"name\":\"read\"}]}"; + + buf_init(&out); + anthropic_build_request(&out, "m", 8, "be brief", msgs, 1, + "[{\"name\":\"read\"}]"); + buf_terminate(&out); + CHECK(strcmp(out.data, expect) == 0, "build sys+tools:\n%s", out.data); + buf_free(&out); +} + +static void +test_build_escaping(void) +{ + struct buf out; + struct anth_msg msgs[] = { { "user", "quote \" and \\ and\nnewline" } }; + const char *expect = + "{\"model\":\"m\",\"max_tokens\":1,\"stream\":true," + "\"messages\":[{\"role\":\"user\"," + "\"content\":\"quote \\\" and \\\\ and\\nnewline\"}]}"; + + buf_init(&out); + anthropic_build_request(&out, "m", 1, NULL, msgs, 1, NULL); + buf_terminate(&out); + CHECK(strcmp(out.data, expect) == 0, "build escaping:\n%s", out.data); + buf_free(&out); +} + +/* ---- streaming event interpreter ---- */ + +static void +test_text_stream(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "content_block_start", + "{\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}"); + ev(&s, "content_block_delta", + "{\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\",\"text\":\"Hello\"}}"); + ev(&s, "content_block_delta", + "{\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\",\"text\":\", world\"}}"); + ev(&s, "content_block_stop", + "{\"type\":\"content_block_stop\",\"index\":0}"); + buf_terminate(&rec_text); + CHECK(strcmp(rec_text.data, "Hello, world") == 0, "text stream: '%s'", + rec_text.data); + CHECK(rec_tools == 0, "text stream: no tool call"); + anthropic_stream_clear(&s); + rec_reset(); +} + +static void +test_tool_call(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "content_block_start", + "{\"type\":\"content_block_start\",\"index\":1," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_1\"," + "\"name\":\"get_weather\",\"input\":{}}}"); + ev(&s, "content_block_delta", + "{\"type\":\"content_block_delta\",\"index\":1," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\"{\\\"x\\\"\"}}"); + ev(&s, "content_block_delta", + "{\"type\":\"content_block_delta\",\"index\":1," + "\"delta\":{\"type\":\"input_json_delta\"," + "\"partial_json\":\":42}\"}}"); + ev(&s, "content_block_stop", + "{\"type\":\"content_block_stop\",\"index\":1}"); + CHECK(rec_tools == 1, "tool: one call (%d)", rec_tools); + CHECK(rec_tool_id && strcmp(rec_tool_id, "toolu_1") == 0, "tool: id"); + CHECK(rec_tool_name && strcmp(rec_tool_name, "get_weather") == 0, + "tool: name"); + CHECK(rec_tool_input && strcmp(rec_tool_input, "{\"x\":42}") == 0, + "tool: input '%s'", rec_tool_input ? rec_tool_input : "(null)"); + anthropic_stream_clear(&s); + rec_reset(); +} + +static void +test_tool_no_args(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "content_block_start", + "{\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"tool_use\",\"id\":\"t2\"," + "\"name\":\"now\",\"input\":{}}}"); + ev(&s, "content_block_stop", + "{\"type\":\"content_block_stop\",\"index\":0}"); + CHECK(rec_tools == 1 && rec_tool_input && + strcmp(rec_tool_input, "{}") == 0, "tool no-args input '%s'", + rec_tool_input ? rec_tool_input : "(null)"); + anthropic_stream_clear(&s); + rec_reset(); +} + +static void +test_usage_stop_done(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "message_start", + "{\"type\":\"message_start\",\"message\":{\"id\":\"m\"," + "\"role\":\"assistant\",\"usage\":{\"input_tokens\":10," + "\"output_tokens\":1}}}"); + CHECK(rec_usage == 1 && rec_in == 10 && rec_out == 1, + "usage start: in=%lld out=%lld", rec_in, rec_out); + ev(&s, "message_delta", + "{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\",\"stop_sequence\":null}," + "\"usage\":{\"output_tokens\":25}}"); + CHECK(rec_stop && strcmp(rec_stop, "end_turn") == 0, "stop reason"); + CHECK(rec_in == 10 && rec_out == 25, "usage delta: in=%lld out=%lld", + rec_in, rec_out); + ev(&s, "message_stop", "{\"type\":\"message_stop\"}"); + CHECK(rec_done == 1, "done"); + anthropic_stream_clear(&s); + rec_reset(); +} + +static void +test_null_stop_reason(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "message_delta", + "{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":null}}"); + CHECK(rec_stop == NULL, "null stop_reason not reported"); + anthropic_stream_clear(&s); + rec_reset(); +} + +static void +test_error_event(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "error", + "{\"type\":\"error\",\"error\":{\"type\":\"overloaded_error\"," + "\"message\":\"slow down\"}}"); + CHECK(rec_errors == 1 && rec_err_type && + strcmp(rec_err_type, "overloaded_error") == 0, "error type"); + CHECK(rec_err_msg && strcmp(rec_err_msg, "slow down") == 0, "error msg"); + /* after a fatal error, further events are ignored */ + ev(&s, "content_block_delta", + "{\"type\":\"content_block_delta\",\"index\":0," + "\"delta\":{\"type\":\"text_delta\",\"text\":\"ignored\"}}"); + CHECK(rec_text.len == 0, "post-error events ignored"); + anthropic_stream_clear(&s); + rec_reset(); +} + +static void +test_ping_and_badjson(void) +{ + struct anthropic_stream s; + + anthropic_stream_init(&s, &RECORDER, NULL); + ev(&s, "ping", "{\"type\":\"ping\"}"); + CHECK(rec_errors == 0 && rec_done == 0 && rec_text.len == 0, + "ping ignored"); + + ev(&s, "content_block_delta", "{"); /* malformed JSON */ + CHECK(rec_errors == 1 && rec_err_type && + strcmp(rec_err_type, "parse_error") == 0, "bad json -> parse_error"); + anthropic_stream_clear(&s); + rec_reset(); +} + +/* ---- integration: SSE wire -> sse -> anthropic ---- */ + +static void +sse_trampoline(const char *event, const char *data, size_t datalen, void *arg) +{ + anthropic_stream_event(arg, event, data, datalen); +} + +static void +test_integration(void) +{ + struct anthropic_stream s; + struct sse_parser p; + const char *wire = + "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"usage\":" + "{\"input_tokens\":7,\"output_tokens\":1}}}\n" + "\n" + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n" + "\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"text_delta\",\"text\":\"It is \"}}\n" + "\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"text_delta\",\"text\":\"sunny.\"}}\n" + "\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n" + "\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\"},\"usage\":{\"output_tokens\":9}}\n" + "\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n" + "\n"; + size_t i, len = strlen(wire); + + anthropic_stream_init(&s, &RECORDER, NULL); + sse_init(&p, sse_trampoline, &s); + + /* feed the wire in 7-byte network-sized pieces */ + for (i = 0; i < len; i += 7) { + size_t n = (len - i < 7) ? len - i : 7; + if (sse_push(&p, wire + i, n) != 0) + failures++; + } + + buf_terminate(&rec_text); + CHECK(strcmp(rec_text.data, "It is sunny.") == 0, + "integration text: '%s'", rec_text.data); + CHECK(rec_in == 7 && rec_out == 9, "integration usage in=%lld out=%lld", + rec_in, rec_out); + CHECK(rec_stop && strcmp(rec_stop, "end_turn") == 0, "integration stop"); + CHECK(rec_done == 1, "integration done"); + + sse_clear(&p); + anthropic_stream_clear(&s); + rec_reset(); +} + +int +main(void) +{ + buf_init(&rec_text); + + test_build_basic(); + test_build_system_tools(); + test_build_escaping(); + test_text_stream(); + test_tool_call(); + test_tool_no_args(); + test_usage_stop_done(); + test_null_stop_reason(); + test_error_event(); + test_ping_and_badjson(); + test_integration(); + + buf_free(&rec_text); + printf("anthropic_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + 171364dcfe0b3bf4079a31790fb59c57bf44d39a (mode 644) --- /dev/null +++ regress/buf/Makefile @@ -0,0 +1,22 @@ +# Unit tests for the growable byte-buffer module (src/common/buf.c). +# Covers the post-audit-hardening contract: buf_terminate writes a NUL but does +# not count it in .len, and buf_cstr's debug-assert catches the missing-NUL bug +# at the read site. + +PROG= buf_test +SRCS= buf_test.c buf.c util.c log.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +LDADD+= -lutil +DPADD+= ${LIBUTIL} + +REGRESS_TARGETS= run-buf_test + +run-buf_test: ${PROG} + ./${PROG} + +.include blob - /dev/null blob + 84c4a5a1b90eafca48034526d47f0ecd922b739d (mode 644) --- /dev/null +++ regress/buf/buf_test.c @@ -0,0 +1,236 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * Unit tests for the buf module, focused on the post-hardening contract: + * + * - buf_terminate(b) writes a '\0' at b->data[b->len] but leaves .len + * pointing at the last logical byte (so strlen(b->data) == b->len). + * - buf_cstr(b) reads .data as a C string and asserts the trailing NUL is + * there; empty buf returns "". + * - buf_append + buf_terminate round-trips bytes including embedded NULs + * (the NUL appears IN the buffer length-prefixed; the trailing NUL is + * the C-string terminator, not part of the payload). + * + * The original bug shape -- buf_puts (now buf_puts_cstr) writes via strlen, + * leaving b->data[b->len] uninitialized -- is documented with a test that + * shows the byte at .len IS unset after just buf_puts_cstr, then becomes + * '\0' after buf_terminate. + */ + +#include + +#include +#include +#include + +#include "buf.h" + +static int checks, failures; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +static void +test_terminate_basic(void) +{ + struct buf b; + + buf_init(&b); + buf_append(&b, "hello", 5); + buf_terminate(&b); + CHECK(b.len == 5, "buf_terminate keeps len at logical bytes (got %zu)", + b.len); + CHECK(b.data[b.len] == '\0', + "buf_terminate writes a NUL at b.data[b.len]"); + CHECK(strlen(b.data) == 5, + "strlen(b.data) matches b.len (got %zu)", strlen(b.data)); + CHECK(strcmp(b.data, "hello") == 0, "payload intact after terminate"); + buf_free(&b); +} + +static void +test_terminate_empty(void) +{ + struct buf b; + + buf_init(&b); + buf_terminate(&b); + CHECK(b.len == 0, "empty + terminate stays at len 0 (got %zu)", b.len); + CHECK(b.data != NULL && b.data[0] == '\0', + "empty + terminate writes NUL at position 0"); + buf_free(&b); +} + +static void +test_buf_cstr_empty(void) +{ + struct buf b; + const char *s; + + buf_init(&b); + /* No allocation -- data is still NULL. */ + s = buf_cstr(&b); + CHECK(s != NULL && *s == '\0', + "buf_cstr on uninitialised buf returns \"\""); + buf_free(&b); +} + +static void +test_buf_cstr_after_terminate(void) +{ + struct buf b; + + buf_init(&b); + buf_puts_cstr(&b, "abcd"); + buf_terminate(&b); + CHECK(strcmp(buf_cstr(&b), "abcd") == 0, + "buf_cstr returns the payload as C string"); + buf_free(&b); +} + +/* + * The original bug: buf_puts_cstr (was buf_puts) writes payload bytes via + * strlen but does NOT write a trailing NUL. After only that call, + * b->data[b->len] is uninitialised heap. buf_terminate fixes it; buf_cstr + * is the read-side contract that catches a missing fix. + */ +static void +test_puts_cstr_does_not_terminate(void) +{ + struct buf b; + + buf_init(&b); + buf_putc(&b, 'X'); /* scribble sentinels at positions 0..2 */ + buf_putc(&b, 'X'); + buf_putc(&b, 'X'); + b.len = 0; /* reset, but the bytes stay in .data */ + + buf_puts_cstr(&b, "ab"); + CHECK(b.len == 2, "buf_puts_cstr sets len to strlen"); + CHECK(b.data[0] == 'a' && b.data[1] == 'b', + "payload bytes written"); + CHECK(b.data[b.len] == 'X', + "b.data[b.len] is the prior sentinel, NOT '\\0' " + "(this is the bug class buf_terminate + buf_cstr exists to catch)"); + + buf_terminate(&b); + CHECK(b.data[b.len] == '\0', + "buf_terminate writes '\\0' at b.data[b.len]"); + CHECK(strcmp(buf_cstr(&b), "ab") == 0, + "buf_cstr round-trips after buf_terminate"); + buf_free(&b); +} + +/* + * Embedded NULs survive buf_append + buf_terminate: the payload is + * length-counted in .len, the trailing terminator is just for safe C-string + * read-back of the AROUND data (it's at b.data[b.len], past the payload). + */ +static void +test_embedded_nul_roundtrip(void) +{ + struct buf b; + const char *payload = "ab\0cd"; /* length 5 incl. NUL */ + size_t i; + int bytes_match = 1; + + buf_init(&b); + buf_append(&b, payload, 5); + buf_terminate(&b); + CHECK(b.len == 5, + "buf_append + buf_terminate preserves length-5 payload " + "(got %zu)", b.len); + for (i = 0; i < 5; i++) + if (b.data[i] != payload[i]) + bytes_match = 0; + CHECK(bytes_match, "bytes (including embedded NUL) match"); + CHECK(b.data[5] == '\0', "trailing NUL after the payload"); + /* + * strlen would return 2 here (it stops at the embedded NUL); that is + * exactly why buf_cstr is documented as dispatch-only when the buf + * may carry NULs, and length-aware readers should use .len directly. + */ + CHECK(strlen(b.data) == 2, "strlen sees only the first NUL"); + buf_free(&b); +} + +/* + * buf_terminate is idempotent in the sense that calling it twice still + * leaves b.data[b.len] == '\0' and b.len unchanged -- two calls are + * surprising but should not corrupt state. Catches a future regression + * that, say, makes buf_terminate forget to decrement .len. + */ +static void +test_terminate_idempotent(void) +{ + struct buf b; + size_t len_before; + + buf_init(&b); + buf_append(&b, "xy", 2); + buf_terminate(&b); + len_before = b.len; + buf_terminate(&b); + CHECK(b.len == len_before, "second buf_terminate keeps .len"); + CHECK(b.data[b.len] == '\0', "trailing NUL still in place"); + CHECK(strcmp(buf_cstr(&b), "xy") == 0, + "buf_cstr still round-trips"); + buf_free(&b); +} + +/* + * buf_freezero should still work after buf_terminate, and the data buffer + * should be cleared to all zeros up to .cap before free. We can only + * observe the pre-free state -- after free the pointer is dangling -- so + * verify by reading the bytes back via b.data (set to NULL by freezero). + */ +static void +test_freezero_after_terminate(void) +{ + struct buf b; + + buf_init(&b); + buf_append(&b, "secret", 6); + buf_terminate(&b); + buf_freezero(&b); + CHECK(b.data == NULL, "buf_freezero NULLs the data pointer"); + CHECK(b.len == 0, "buf_freezero zeros the length"); + CHECK(b.cap == 0, "buf_freezero zeros the capacity"); +} + +int +main(void) +{ + test_terminate_basic(); + test_terminate_empty(); + test_buf_cstr_empty(); + test_buf_cstr_after_terminate(); + test_puts_cstr_does_not_terminate(); + test_embedded_nul_roundtrip(); + test_terminate_idempotent(); + test_freezero_after_terminate(); + + printf("buf_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + 581967c540081b97e030687f396e67eab011aa41 (mode 644) --- /dev/null +++ regress/chat/Makefile @@ -0,0 +1,34 @@ +# Tracer-bullet end-to-end test: run fugu(1) chatting with a libtls stub server. +# Requires the worker binaries to be built (run `make` at the top level first). + +PROG= chat_test +SRCS= chat_test.c buf.c util.c log.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +LDADD+= -ltls -lutil +DPADD+= ${LIBTLS} ${LIBUTIL} + +FUGU_BIN= ${.CURDIR}/../../src/parent/fugu +FUGU_UI_BIN= ${.CURDIR}/../../src/ui/fugu-ui +FUGU_NET_BIN= ${.CURDIR}/../../src/net/fugu-net +FUGU_EXEC_BIN= ${.CURDIR}/../../src/exec/fugu-exec +FUGU_FETCH_BIN= ${.CURDIR}/../../src/fetch/fugu-fetch + +REGRESS_TARGETS= run-chat_test + +run-chat_test: ${PROG} + @td=`mktemp -d /tmp/fugu-chat-cert.XXXXXX`; \ + openssl req -x509 -newkey rsa:2048 -nodes \ + -keyout $$td/key.pem -out $$td/cert.pem \ + -days 1 -subj '/CN=localhost' >/dev/null 2>&1; \ + FUGU_BIN=${FUGU_BIN} FUGU_UI_BIN=${FUGU_UI_BIN} \ + FUGU_NET_BIN=${FUGU_NET_BIN} FUGU_EXEC_BIN=${FUGU_EXEC_BIN} \ + FUGU_FETCH_BIN=${FUGU_FETCH_BIN} \ + ./${PROG} $$td/cert.pem $$td/key.pem; rc=$$?; \ + rm -rf $$td; exit $$rc + +.include blob - /dev/null blob + e1ae12a6ec0886e822d476c1a5790cde4d563aeb (mode 644) --- /dev/null +++ regress/chat/chat_test.c @@ -0,0 +1,326 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * The tracer bullet, end to end: run the real `fugu` binary (which spawns + * fugu-ui and fugu-net), feed a line to its stdin, and read the streamed reply + * from its stdout. net is pointed at a libtls stub server standing in for + * api.anthropic.com, so no credentials or outbound network are needed. + * + * Requires FUGU_BIN / FUGU_UI_BIN / FUGU_NET_BIN in the environment (set by the + * Makefile) and a cert/key pair in argv (generated by the Makefile). + */ + +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "buf.h" + +static int checks; +static int failures; + +/* Remove the session journals the run leaves behind, then the directory. */ +static void +rm_sessions(const char *home) +{ + char path[1100], f[1200]; + DIR *d; + struct dirent *e; + + snprintf(path, sizeof(path), "%s/.fugu/sessions", home); + if ((d = opendir(path)) != NULL) { + while ((e = readdir(d)) != NULL) { + if (e->d_name[0] == '.') + continue; + snprintf(f, sizeof(f), "%s/%s", path, e->d_name); + unlink(f); + } + closedir(d); + } + rmdir(path); +} + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +static const char *const SSE_BODY = + "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n" + "\n" + "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"text_delta\",\"text\":\"pong\"}}\n" + "\n" + "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n" + "\n" + "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\"},\"usage\":{\"output_tokens\":4}}\n" + "\n" + "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n" + "\n"; + +static void +add_chunk(struct buf *b, const char *data, size_t n) +{ + buf_printf(b, "%zx\r\n", n); + buf_append(b, data, n); + buf_append(b, "\r\n", 2); +} + +static int +tls_write_all(struct tls *t, const void *buf, size_t len) +{ + const unsigned char *p = buf; + size_t off = 0; + ssize_t n; + + while (off < len) { + n = tls_write(t, p + off, len - off); + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) + continue; + if (n < 0) + return (-1); + off += (size_t)n; + } + return (0); +} + +static void +stub_server(int lfd, const char *cert, const char *key) +{ + struct tls_config *cfg; + struct tls *srv, *cctx; + struct buf resp; + char req[2048]; + ssize_t n; + int cfd; + size_t sl, half; + + if ((cfd = accept(lfd, NULL, NULL)) == -1) + _exit(20); + if ((cfg = tls_config_new()) == NULL) + _exit(21); + if (tls_config_set_cert_file(cfg, cert) == -1 || + tls_config_set_key_file(cfg, key) == -1) + _exit(22); + if ((srv = tls_server()) == NULL || tls_configure(srv, cfg) == -1) + _exit(23); + if (tls_accept_socket(srv, &cctx, cfd) == -1) + _exit(24); + do { + n = tls_handshake(cctx); + } while (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT); + if (n == -1) + _exit(25); + do { + n = tls_read(cctx, req, sizeof(req)); + } while (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT); + + buf_init(&resp); + buf_puts_cstr(&resp, "HTTP/1.1 200 OK\r\n"); + buf_puts_cstr(&resp, "Content-Type: text/event-stream\r\n"); + buf_puts_cstr(&resp, "Transfer-Encoding: chunked\r\n\r\n"); + sl = strlen(SSE_BODY); + half = sl / 2; + add_chunk(&resp, SSE_BODY, half); + add_chunk(&resp, SSE_BODY + half, sl - half); + buf_puts_cstr(&resp, "0\r\n\r\n"); + (void)tls_write_all(cctx, resp.data, resp.len); + buf_free(&resp); + tls_close(cctx); + _exit(0); +} + +static void +write_config(const char *home) +{ + char dir[1024], path[1024]; + FILE *fp; + int fd; + + snprintf(dir, sizeof(dir), "%s/.fugu", home); + if (mkdir(dir, 0700) == -1) + err(1, "mkdir %s", dir); + snprintf(path, sizeof(path), "%s/.fugu/fugu.conf", home); + if ((fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0600)) == -1) + err(1, "open %s", path); + if (fchmod(fd, 0600) == -1) /* defeat a permissive umask */ + err(1, "fchmod"); + if ((fp = fdopen(fd, "w")) == NULL) + err(1, "fdopen"); + fputs("anthropic_key \"test-key\"\n", fp); + fputs("model \"claude-sonnet-4-6\"\n", fp); + fputs("max_tokens 64\n", fp); + fclose(fp); +} + +static void +on_alarm(int sig) +{ + (void)sig; + dprintf(STDERR_FILENO, "chat_test: TIMED OUT\n"); + _exit(97); +} + +int +main(int argc, char *argv[]) +{ + struct sockaddr_in sa; + socklen_t slen; + pid_t sp, fp; + const char *fugu_bin; + char home[] = "/tmp/fugu-chat.XXXXXX"; + char out[8192], portstr[16]; + size_t outlen = 0; + ssize_t k; + int lfd, one = 1, port, status; + int in_pipe[2], out_pipe[2]; + + if (argc != 3) + errx(1, "usage: %s cert.pem key.pem", argv[0]); + if ((fugu_bin = getenv("FUGU_BIN")) == NULL) + errx(1, "FUGU_BIN not set"); + + signal(SIGALRM, on_alarm); + signal(SIGPIPE, SIG_IGN); + alarm(90); + + if (mkdtemp(home) == NULL) + err(1, "mkdtemp"); + write_config(home); + + /* stub-server TCP listener on an ephemeral 127.0.0.1 port */ + if ((lfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) + err(1, "socket"); + setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)); + memset(&sa, 0, sizeof(sa)); + sa.sin_family = AF_INET; + sa.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + if (bind(lfd, (struct sockaddr *)&sa, sizeof(sa)) == -1) + err(1, "bind"); + if (listen(lfd, 1) == -1) + err(1, "listen"); + slen = sizeof(sa); + if (getsockname(lfd, (struct sockaddr *)&sa, &slen) == -1) + err(1, "getsockname"); + port = ntohs(sa.sin_port); + snprintf(portstr, sizeof(portstr), "%d", port); + + if ((sp = fork()) == -1) + err(1, "fork"); + if (sp == 0) { + stub_server(lfd, argv[1], argv[2]); + _exit(0); + } + close(lfd); + + if (pipe(in_pipe) == -1 || pipe(out_pipe) == -1) + err(1, "pipe"); + + if ((fp = fork()) == -1) + err(1, "fork"); + if (fp == 0) { + dup2(in_pipe[0], STDIN_FILENO); + dup2(out_pipe[1], STDOUT_FILENO); + close(in_pipe[0]); + close(in_pipe[1]); + close(out_pipe[0]); + close(out_pipe[1]); + setenv("HOME", home, 1); + { /* not setgid in the harness, so FUGU_CONF is honoured */ + char confpath[1100]; + + snprintf(confpath, sizeof(confpath), "%s/.fugu/fugu.conf", + home); + setenv("FUGU_CONF", confpath, 1); + } + setenv("FUGU_NET_HOST", "localhost", 1); + setenv("FUGU_NET_PORT", portstr, 1); + setenv("FUGU_NET_CAFILE", argv[1], 1); + execl(fugu_bin, fugu_bin, (char *)NULL); + _exit(127); + } + close(in_pipe[0]); + close(out_pipe[1]); + + /* drive the conversation */ + if (write(in_pipe[1], "ping\n", 5) != 5) + CHECK(0, "write to fugu stdin"); + + while (outlen < sizeof(out) - 1) { + k = read(out_pipe[0], out + outlen, sizeof(out) - 1 - outlen); + if (k <= 0) + break; + outlen += (size_t)k; + out[outlen] = '\0'; + if (strstr(out, "pong") != NULL) + break; + } + CHECK(strstr(out, "pong") != NULL, "assistant reply contains 'pong'"); + + close(in_pipe[1]); /* EOF -> ui quits -> fugu shuts down */ + close(out_pipe[0]); + + if (waitpid(fp, &status, 0) == -1) + err(1, "waitpid fugu"); + CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0, + "fugu exit status %d", status); + if (waitpid(sp, &status, 0) == -1) + err(1, "waitpid stub"); + CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0, + "stub server exit %d", status); + + /* best-effort cleanup of the temp HOME tree */ + { + char p[1100]; + + snprintf(p, sizeof(p), "%s/.fugu/fugu.conf", home); + unlink(p); + rm_sessions(home); + snprintf(p, sizeof(p), "%s/.fugu", home); + rmdir(p); + rmdir(home); + } + + printf("chat_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + fa935eb6a4497a82ce29e2f5545fc9c956f7c825 (mode 644) --- /dev/null +++ regress/compact/Makefile @@ -0,0 +1,34 @@ +# End-to-end test of /compact against a libtls stub api.anthropic.com. +# Requires the worker binaries to be built (run `make` at the top level first). + +PROG= compact_test +SRCS= compact_test.c buf.c util.c log.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +LDADD+= -ltls -lutil +DPADD+= ${LIBTLS} ${LIBUTIL} + +FUGU_BIN= ${.CURDIR}/../../src/parent/fugu +FUGU_UI_BIN= ${.CURDIR}/../../src/ui/fugu-ui +FUGU_NET_BIN= ${.CURDIR}/../../src/net/fugu-net +FUGU_EXEC_BIN= ${.CURDIR}/../../src/exec/fugu-exec +FUGU_FETCH_BIN= ${.CURDIR}/../../src/fetch/fugu-fetch + +REGRESS_TARGETS= run-compact_test + +run-compact_test: ${PROG} + @td=`mktemp -d /tmp/fugu-compact-cert.XXXXXX`; \ + openssl req -x509 -newkey rsa:2048 -nodes \ + -keyout $$td/key.pem -out $$td/cert.pem \ + -days 1 -subj '/CN=localhost' >/dev/null 2>&1; \ + FUGU_BIN=${FUGU_BIN} FUGU_UI_BIN=${FUGU_UI_BIN} \ + FUGU_NET_BIN=${FUGU_NET_BIN} FUGU_EXEC_BIN=${FUGU_EXEC_BIN} \ + FUGU_FETCH_BIN=${FUGU_FETCH_BIN} \ + ./${PROG} $$td/cert.pem $$td/key.pem; rc=$$?; \ + rm -rf $$td; exit $$rc + +.include blob - /dev/null blob + 38bf6abb445f3e36e9c450325af2af19a4317b52 (mode 644) --- /dev/null +++ regress/compact/compact_test.c @@ -0,0 +1,398 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * End-to-end test of /compact: run the real fugu (line mode) against a libtls + * stub standing in for api.anthropic.com. The stub serves two connections -- a + * normal turn ("pong") and then the summarisation call ("Compacted summary."). + * We type "ping" then "/compact" and verify the summary is shown and that the + * journal records a compact marker carrying the summary text. + */ + +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "buf.h" + +static int checks, failures; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +/* Append a complete single-text-block SSE response with the given text. */ +static void +make_sse(struct buf *b, const char *text) +{ + buf_puts_cstr(b, "event: message_start\n" + "data: {\"type\":\"message_start\",\"message\":{\"id\":\"x\"," + "\"role\":\"assistant\",\"usage\":{\"input_tokens\":5," + "\"output_tokens\":0}}}\n\n"); + buf_puts_cstr(b, "event: content_block_start\n" + "data: {\"type\":\"content_block_start\",\"index\":0," + "\"content_block\":{\"type\":\"text\",\"text\":\"\"}}\n\n"); + buf_printf(b, "event: content_block_delta\n" + "data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":" + "{\"type\":\"text_delta\",\"text\":\"%s\"}}\n\n", text); + buf_puts_cstr(b, "event: content_block_stop\n" + "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n"); + buf_puts_cstr(b, "event: message_delta\n" + "data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":" + "\"end_turn\"},\"usage\":{\"output_tokens\":4}}\n\n"); + buf_puts_cstr(b, "event: message_stop\n" + "data: {\"type\":\"message_stop\"}\n\n"); +} + +static void +add_chunk(struct buf *b, const char *data, size_t n) +{ + buf_printf(b, "%zx\r\n", n); + buf_append(b, data, n); + buf_append(b, "\r\n", 2); +} + +static int +tls_write_all(struct tls *t, const void *buf, size_t len) +{ + const unsigned char *p = buf; + size_t off = 0; + ssize_t n; + + while (off < len) { + n = tls_write(t, p + off, len - off); + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) + continue; + if (n < 0) + return (-1); + off += (size_t)n; + } + return (0); +} + +/* Accept one TLS connection, read the request, and stream `text` as SSE. */ +static void +serve_one(int lfd, struct tls *srv, const char *text) +{ + struct tls *cctx; + struct buf body, resp; + char req[4096]; + ssize_t n; + int cfd; + size_t sl, half; + + if ((cfd = accept(lfd, NULL, NULL)) == -1) + _exit(30); + if (tls_accept_socket(srv, &cctx, cfd) == -1) + _exit(31); + do { + n = tls_handshake(cctx); + } while (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT); + if (n == -1) + _exit(32); + do { + n = tls_read(cctx, req, sizeof(req)); + } while (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT); + + buf_init(&body); + make_sse(&body, text); + + buf_init(&resp); + buf_puts_cstr(&resp, "HTTP/1.1 200 OK\r\n"); + buf_puts_cstr(&resp, "Content-Type: text/event-stream\r\n"); + buf_puts_cstr(&resp, "Transfer-Encoding: chunked\r\n\r\n"); + sl = body.len; + half = sl / 2; + add_chunk(&resp, body.data, half); + add_chunk(&resp, body.data + half, sl - half); + buf_puts_cstr(&resp, "0\r\n\r\n"); + (void)tls_write_all(cctx, resp.data, resp.len); + buf_free(&resp); + buf_free(&body); + tls_close(cctx); + close(cfd); +} + +static void +stub_server(int lfd, const char *cert, const char *key) +{ + struct tls_config *cfg; + struct tls *srv; + + if ((cfg = tls_config_new()) == NULL) + _exit(21); + if (tls_config_set_cert_file(cfg, cert) == -1 || + tls_config_set_key_file(cfg, key) == -1) + _exit(22); + if ((srv = tls_server()) == NULL || tls_configure(srv, cfg) == -1) + _exit(23); + + serve_one(lfd, srv, "pong"); /* the normal turn */ + serve_one(lfd, srv, "Compacted summary."); /* the /compact call */ + + tls_free(srv); + _exit(0); +} + +static void +write_config(const char *home) +{ + char dir[1024], path[1024]; + FILE *fp; + int fd; + + snprintf(dir, sizeof(dir), "%s/.fugu", home); + if (mkdir(dir, 0700) == -1) + err(1, "mkdir %s", dir); + snprintf(dir, sizeof(dir), "%s/.fugu/sessions", home); + if (mkdir(dir, 0700) == -1) + err(1, "mkdir %s", dir); + snprintf(path, sizeof(path), "%s/.fugu/fugu.conf", home); + if ((fd = open(path, O_CREAT | O_WRONLY | O_TRUNC, 0600)) == -1) + err(1, "open %s", path); + if (fchmod(fd, 0600) == -1) + err(1, "fchmod"); + if ((fp = fdopen(fd, "w")) == NULL) + err(1, "fdopen"); + fputs("anthropic_key \"test-key\"\n", fp); + fputs("model \"claude-sonnet-4-6\"\n", fp); + fputs("max_tokens 64\n", fp); + fclose(fp); +} + +/* Read the single session journal in HOME into buf (NUL-terminated). */ +static void +read_journal(const char *home, char *out, size_t outsz) +{ + char dir[1100], path[2300]; + DIR *d; + struct dirent *e; + FILE *fp; + size_t n = 0; + + out[0] = '\0'; + snprintf(dir, sizeof(dir), "%s/.fugu/sessions", home); + if ((d = opendir(dir)) == NULL) + return; + while ((e = readdir(d)) != NULL) { + if (e->d_name[0] == '.') + continue; + snprintf(path, sizeof(path), "%s/%s", dir, e->d_name); + if ((fp = fopen(path, "r")) != NULL) { + n += fread(out + n, 1, outsz - 1 - n, fp); + fclose(fp); + } + } + closedir(d); + out[n] = '\0'; +} + +static void +rm_sessions(const char *home) +{ + char path[1100], f[2300]; + DIR *d; + struct dirent *e; + + snprintf(path, sizeof(path), "%s/.fugu/sessions", home); + if ((d = opendir(path)) != NULL) { + while ((e = readdir(d)) != NULL) { + if (e->d_name[0] == '.') + continue; + snprintf(f, sizeof(f), "%s/%s", path, e->d_name); + unlink(f); + } + closedir(d); + } + rmdir(path); +} + +static void +on_alarm(int sig) +{ + (void)sig; + dprintf(STDERR_FILENO, "compact_test: TIMED OUT\n"); + _exit(97); +} + +/* Read from fd into out (appending), stopping when needle appears or EOF. */ +static int +read_until(int fd, char *out, size_t *len, size_t cap, const char *needle) +{ + ssize_t k; + + while (*len < cap - 1) { + if (strstr(out, needle) != NULL) + return (1); + k = read(fd, out + *len, cap - 1 - *len); + if (k <= 0) + break; + *len += (size_t)k; + out[*len] = '\0'; + } + return (strstr(out, needle) != NULL); +} + +int +main(int argc, char *argv[]) +{ + struct sockaddr_in sa; + socklen_t slen; + pid_t sp, fp; + const char *fugu_bin; + char home[] = "/tmp/fugu-compact.XXXXXX"; + char out[16384], jrnl[16384], portstr[16]; + size_t outlen = 0; + int lfd, one = 1, port, status; + int in_pipe[2], out_pipe[2]; + + if (argc != 3) + errx(1, "usage: %s cert.pem key.pem", argv[0]); + if ((fugu_bin = getenv("FUGU_BIN")) == NULL) + errx(1, "FUGU_BIN not set"); + + signal(SIGALRM, on_alarm); + signal(SIGPIPE, SIG_IGN); + alarm(90); + + if (mkdtemp(home) == NULL) + err(1, "mkdtemp"); + write_config(home); + + if ((lfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) + err(1, "socket"); + setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)); + memset(&sa, 0, sizeof(sa)); + sa.sin_family = AF_INET; + sa.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + if (bind(lfd, (struct sockaddr *)&sa, sizeof(sa)) == -1) + err(1, "bind"); + if (listen(lfd, 2) == -1) + err(1, "listen"); + slen = sizeof(sa); + if (getsockname(lfd, (struct sockaddr *)&sa, &slen) == -1) + err(1, "getsockname"); + port = ntohs(sa.sin_port); + snprintf(portstr, sizeof(portstr), "%d", port); + + if ((sp = fork()) == -1) + err(1, "fork"); + if (sp == 0) { + stub_server(lfd, argv[1], argv[2]); + _exit(0); + } + close(lfd); + + if (pipe(in_pipe) == -1 || pipe(out_pipe) == -1) + err(1, "pipe"); + + if ((fp = fork()) == -1) + err(1, "fork"); + if (fp == 0) { + dup2(in_pipe[0], STDIN_FILENO); + dup2(out_pipe[1], STDOUT_FILENO); + close(in_pipe[0]); + close(in_pipe[1]); + close(out_pipe[0]); + close(out_pipe[1]); + setenv("HOME", home, 1); + { /* not setgid in the harness, so FUGU_CONF is honoured */ + char confpath[1100]; + + snprintf(confpath, sizeof(confpath), "%s/.fugu/fugu.conf", + home); + setenv("FUGU_CONF", confpath, 1); + } + setenv("FUGU_NET_HOST", "localhost", 1); + setenv("FUGU_NET_PORT", portstr, 1); + setenv("FUGU_NET_CAFILE", argv[1], 1); + execl(fugu_bin, fugu_bin, (char *)NULL); + _exit(127); + } + close(in_pipe[0]); + close(out_pipe[1]); + + out[0] = '\0'; + + /* A normal turn first, to give /compact something to summarise. */ + if (write(in_pipe[1], "ping\n", 5) != 5) + CHECK(0, "write ping"); + CHECK(read_until(out_pipe[0], out, &outlen, sizeof(out), "pong"), + "normal turn streamed 'pong'"); + + /* Now compact the conversation. */ + if (write(in_pipe[1], "/compact\n", 9) != 9) + CHECK(0, "write /compact"); + CHECK(read_until(out_pipe[0], out, &outlen, sizeof(out), + "Compacted summary."), "compact summary shown to the user"); + CHECK(strstr(out, "compacted") != NULL, "compact divider shown"); + + close(in_pipe[1]); /* EOF -> ui quits -> fugu shuts down */ + while (read(out_pipe[0], out + outlen, + outlen < sizeof(out) - 1 ? sizeof(out) - 1 - outlen : 0) > 0) + ; /* drain */ + close(out_pipe[0]); + + if (waitpid(fp, &status, 0) == -1) + err(1, "waitpid fugu"); + CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0, + "fugu exit status %d", status); + if (waitpid(sp, &status, 0) == -1) + err(1, "waitpid stub"); + CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0, + "stub server exit %d", status); + + /* The journal must record the compact marker carrying the summary. */ + read_journal(home, jrnl, sizeof(jrnl)); + CHECK(strstr(jrnl, "\"fugu\":\"compact\"") != NULL, + "journal has a compact marker"); + CHECK(strstr(jrnl, "Compacted summary.") != NULL, + "compact marker carries the summary text"); + + { + char p[1100]; + + snprintf(p, sizeof(p), "%s/.fugu/fugu.conf", home); + unlink(p); + rm_sessions(home); + snprintf(p, sizeof(p), "%s/.fugu", home); + rmdir(p); + rmdir(home); + } + + printf("compact_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + 5bdf2ee1a8d700b74d7cd8be68c8d376bb75262a (mode 644) --- /dev/null +++ regress/conf/Makefile @@ -0,0 +1,20 @@ +# Unit tests for config resolution, scoping, and StrictModes (src/common/conf.c). +# The parse.y grammar gets its own regress once it lands. + +PROG= conf_test +SRCS= conf_test.c conf.c util.c log.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +LDADD+= -lutil +DPADD+= ${LIBUTIL} + +REGRESS_TARGETS= run-conf_test + +run-conf_test: ${PROG} + ./${PROG} + +.include blob - /dev/null blob + 775771f25e42fe29fa4e4e7c5361baff92a4e7b8 (mode 644) --- /dev/null +++ regress/conf/conf_test.c @@ -0,0 +1,335 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include +#include +#include +#include +#include + +#include "conf.h" + +static int fails; +static int checks; + +#define CHECK(cond, msg) do { \ + checks++; \ + if (!(cond)) { \ + fprintf(stderr, "FAIL %s:%d: %s\n", \ + __func__, __LINE__, (msg)); \ + fails++; \ + } \ +} while (0) + +static void +streq(const char *func, int line, const char *got, const char *want, + const char *msg) +{ + checks++; + if (got == NULL || strcmp(got, want) != 0) { + fprintf(stderr, "FAIL %s:%d: %s (got %s want %s)\n", + func, line, msg, got ? got : "(null)", want); + fails++; + } +} +#define STREQ(got, want, msg) streq(__func__, __LINE__, (got), (want), (msg)) + +/* Build a representative config: + * global: model sonnet, max_tokens 8192, allow_write no + * match user isaac: model opus, allow_write yes + * match group interns: model haiku, web_search no + * match user isaac (2nd): model loser <- must never win (first-match-wins) + */ +static void +build(struct conf *c) +{ + struct conf_match *m; + + conf_init(c); + + conf_set_string(&c->global, CS_MODEL, &c->global.model, "sonnet"); + c->global.max_tokens = 8192; + conf_set_flag(&c->global, CS_MAX_TOKENS); + c->global.allow_write = 0; + conf_set_flag(&c->global, CS_ALLOW_WRITE); + + m = conf_match_new(c, MATCH_USER, "isaac"); + conf_set_string(&m->settings, CS_MODEL, &m->settings.model, "opus"); + m->settings.allow_write = 1; + conf_set_flag(&m->settings, CS_ALLOW_WRITE); + + m = conf_match_new(c, MATCH_GROUP, "interns"); + conf_set_string(&m->settings, CS_MODEL, &m->settings.model, "haiku"); + m->settings.web_search = 0; + conf_set_flag(&m->settings, CS_WEB_SEARCH); + + m = conf_match_new(c, MATCH_USER, "isaac"); + conf_set_string(&m->settings, CS_MODEL, &m->settings.model, "loser"); +} + +static void +test_user_match(void) +{ + struct conf c; + struct conf_settings eff; + + build(&c); + memset(&eff, 0, sizeof(eff)); + conf_resolve(&c, "isaac", NULL, 0, &eff); + + STREQ(eff.model, "opus", "user block overrides global, first wins"); + CHECK((eff.set & CS_ALLOW_WRITE) && eff.allow_write == 1, + "user block sets allow_write"); + CHECK((eff.set & CS_MAX_TOKENS) && eff.max_tokens == 8192, + "global max_tokens inherited"); + CHECK(!(eff.set & CS_WEB_SEARCH), "web_search left unset"); + + conf_settings_clear(&eff); + conf_clear(&c); +} + +static void +test_group_match(void) +{ + struct conf c; + struct conf_settings eff; + char *groups[] = { "wheel", "interns" }; + + build(&c); + memset(&eff, 0, sizeof(eff)); + conf_resolve(&c, "bob", groups, 2, &eff); + + STREQ(eff.model, "haiku", "group block matched"); + CHECK((eff.set & CS_WEB_SEARCH) && eff.web_search == 0, + "group block sets web_search no"); + CHECK((eff.set & CS_ALLOW_WRITE) && eff.allow_write == 0, + "allow_write falls back to global"); + + conf_settings_clear(&eff); + conf_clear(&c); +} + +static void +test_no_match(void) +{ + struct conf c; + struct conf_settings eff; + + build(&c); + memset(&eff, 0, sizeof(eff)); + conf_resolve(&c, "carol", NULL, 0, &eff); + + STREQ(eff.model, "sonnet", "no block matches -> global default"); + CHECK(eff.allow_write == 0, "global allow_write"); + + conf_settings_clear(&eff); + conf_clear(&c); +} + +static void +test_print_redacts(void) +{ + struct conf_settings s; + char *buf = NULL; + size_t sz = 0; + FILE *fp; + + memset(&s, 0, sizeof(s)); + conf_set_string(&s, CS_MODEL, &s.model, "opus"); + conf_set_string(&s, CS_SYSTEM, &s.system, "be terse"); + conf_set_string(&s, CS_ANTHROPIC_KEY, &s.anthropic_key, "sk-TOPSECRET"); + conf_set_string(&s, CS_PROVIDER, &s.provider, "openai"); + conf_set_string(&s, CS_API_HOST, &s.api_host, "openrouter.ai"); + conf_set_string(&s, CS_API_PATH, &s.api_path, "/api/v1/chat/completions"); + conf_set_string(&s, CS_API_KEY, &s.api_key, "sk-or-TOPSECRET"); + + fp = open_memstream(&buf, &sz); + if (fp == NULL) { + CHECK(0, "open_memstream"); + return; + } + conf_settings_print(&s, fp); + fclose(fp); + + CHECK(strstr(buf, "model \"opus\"") != NULL, "print shows model"); + CHECK(strstr(buf, "system \"be terse\"") != NULL, "print shows system"); + CHECK(strstr(buf, "provider \"openai\"") != NULL, "print shows provider"); + CHECK(strstr(buf, "api_host \"openrouter.ai\"") != NULL, + "print shows api_host"); + CHECK(strstr(buf, "api_path \"/api/v1/chat/completions\"") != NULL, + "print shows api_path"); + CHECK(strstr(buf, "sk-TOPSECRET") == NULL, "secret value not printed"); + CHECK(strstr(buf, "sk-or-TOPSECRET") == NULL, "api_key value not printed"); + CHECK(strstr(buf, "***") != NULL, "secret redaction marker present"); + + free(buf); + conf_settings_clear(&s); +} + +static void +test_secrecy(void) +{ + char tmpl[] = "/tmp/fugu_secrecy.XXXXXX"; + int fd; + uid_t me = getuid(); + + fd = mkstemp(tmpl); + CHECK(fd != -1, "mkstemp"); + if (fd == -1) + return; + + CHECK(fchmod(fd, 0600) == 0, "fchmod 0600"); + CHECK(conf_check_secrecy(fd, tmpl, me) == 0, "0600 owned by us is ok"); + + /* Group read is allowed: that is how the setgid _fugu group reads it. */ + CHECK(fchmod(fd, 0640) == 0, "fchmod 0640"); + CHECK(conf_check_secrecy(fd, tmpl, me) == 0, "0640 (group read) is ok"); + + CHECK(fchmod(fd, 0644) == 0, "fchmod 0644"); + CHECK(conf_check_secrecy(fd, tmpl, me) == -1, "0644 (world read) rejected"); + + CHECK(fchmod(fd, 0660) == 0, "fchmod 0660"); + CHECK(conf_check_secrecy(fd, tmpl, me) == -1, "0660 (group write) rejected"); + + CHECK(fchmod(fd, 0600) == 0, "fchmod 0600 again"); + CHECK(conf_check_secrecy(fd, tmpl, me + 1) == -1, + "owner neither root nor us rejected"); + + close(fd); + unlink(tmpl); +} + +/* + * Local mirror of src/parent/parent.c:resolve_secret() (file-path branch only). + * resolve_secret() is static in parent.c and the helper cannot be linked here + * without pulling in the parent's worker/imsg/json plumbing. Keep this in + * sync with the production helper -- this test guards the empty-key-file + * regression where an empty first line was silently returned as a zero-length + * key, bypassing the caller's NULL check and disabling provider auth. + */ +static char * +resolve_secret_file(const char *file_path) +{ + FILE *fp; + char *line = NULL; + size_t cap = 0; + ssize_t n; + + if ((fp = fopen(file_path, "r")) == NULL) + return (NULL); + n = getline(&line, &cap, fp); + fclose(fp); + if (n <= 0) { + free(line); + return (NULL); + } + while (n > 0 && (line[n - 1] == '\n' || line[n - 1] == '\r')) + line[--n] = '\0'; + if (n == 0) { + free(line); + return (NULL); + } + return (line); +} + +/* Create a fresh temp key file with the given contents, return the path. */ +static char * +make_keyfile(const char *content, size_t len) +{ + static const char PATTERN[] = "/tmp/fugu_keyfile.XXXXXX"; + char *tmpl = strdup(PATTERN); + int fd; + + if (tmpl == NULL) + return (NULL); + fd = mkstemp(tmpl); + if (fd == -1) { + free(tmpl); + return (NULL); + } + if (len > 0) + (void)write(fd, content, len); + close(fd); + return (tmpl); +} + +static void +test_resolve_secret_empty(void) +{ + char *path, *got; + + /* Truly empty file: getline() returns -1, must yield NULL. */ + path = make_keyfile("", 0); + CHECK(path != NULL, "mkstemp empty"); + if (path != NULL) { + got = resolve_secret_file(path); + CHECK(got == NULL, "empty key file -> NULL (no bytes)"); + free(got); + unlink(path); + free(path); + } + + /* File containing only a newline: getline() returns 1, trim leaves + * length 0; the fix must convert this to NULL rather than returning + * a zero-length string. */ + path = make_keyfile("\n", 1); + CHECK(path != NULL, "mkstemp newline"); + if (path != NULL) { + got = resolve_secret_file(path); + CHECK(got == NULL, "key file with only newline -> NULL"); + free(got); + unlink(path); + free(path); + } + + /* CRLF-only file: same expectation. */ + path = make_keyfile("\r\n", 2); + CHECK(path != NULL, "mkstemp CRLF"); + if (path != NULL) { + got = resolve_secret_file(path); + CHECK(got == NULL, "key file with only CRLF -> NULL"); + free(got); + unlink(path); + free(path); + } + + /* Non-empty file: must still return the trimmed contents. */ + path = make_keyfile("sk-abc\n", 7); + CHECK(path != NULL, "mkstemp non-empty"); + if (path != NULL) { + got = resolve_secret_file(path); + STREQ(got, "sk-abc", "non-empty key file returns trimmed key"); + free(got); + unlink(path); + free(path); + } +} + +int +main(void) +{ + test_user_match(); + test_group_match(); + test_no_match(); + test_print_redacts(); + test_secrecy(); + test_resolve_secret_empty(); + + printf("%d checks, %d failures\n", checks, fails); + return (fails != 0); +} blob - /dev/null blob + 71237af4e1757c39acc15151488483fd7e1a7664 (mode 644) --- /dev/null +++ regress/exec/Makefile @@ -0,0 +1,22 @@ +# Integration test for the exec tool worker (exec_run.c, tools_file.c, +# tools_shell.c). + +PROG= exec_test +SRCS= exec_test.c exec_run.c tools_file.c tools_shell.c \ + json.c buf.c util.c log.c imsgproto.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +EXECDIR= ${.CURDIR}/../../src/exec +CFLAGS+= -I${COMMONDIR} -I${EXECDIR} +.PATH: ${COMMONDIR} ${EXECDIR} + +LDADD+= -lutil +DPADD+= ${LIBUTIL} + +REGRESS_TARGETS= run-exec_test + +run-exec_test: ${PROG} + ./${PROG} + +.include blob - /dev/null blob + 38318ee15b719f0604141db433fb779e7624cf2e (mode 644) --- /dev/null +++ regress/exec/exec_test.c @@ -0,0 +1,337 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * Drives the exec worker over imsg: a forked, sandboxed exec_loop in a temp + * project dir, exercised through M_TOOL_REQUEST/M_TOOL_RESULT. Verifies + * read/write/edit behaviour and file effects, the error cases (not found, not + * unique, missing file, unknown tool), the allow_write gate, and that a write + * outside the unveil'd cwd is refused. + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "buf.h" +#include "exec_run.h" +#include "imsgproto.h" +#include "json.h" + +static int checks; +static int failures; +static char tmpdir[256]; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +static void +writefile(const char *name, const char *content) +{ + char p[512]; + int fd; + + snprintf(p, sizeof(p), "%s/%s", tmpdir, name); + if ((fd = open(p, O_CREAT | O_WRONLY | O_TRUNC, 0644)) == -1) + err(1, "open %s", p); + if (write(fd, content, strlen(content)) != (ssize_t)strlen(content)) + err(1, "write %s", p); + close(fd); +} + +static int +readfile(const char *name, char *buf, size_t cap) +{ + char p[512]; + ssize_t n; + int fd; + + snprintf(p, sizeof(p), "%s/%s", tmpdir, name); + if ((fd = open(p, O_RDONLY)) == -1) + return (-1); + n = read(fd, buf, cap - 1); + close(fd); + if (n < 0) + return (-1); + buf[n] = '\0'; + return ((int)n); +} + +static int +recv_msg(struct imsgproto *ip, struct fugu_msg *m) +{ + int r, n; + + for (;;) { + if ((r = ip_get(ip, m)) == 1) + return (1); + if (r == -1) + return (-1); + if ((n = ip_read(ip)) <= 0) + return (0); + } +} + +static void +send_req(struct imsgproto *ip, const char *id, const char *name, + const char *input_json) +{ + struct json_writer w; + struct buf b; + + buf_init(&b); + json_writer_init(&w, &b); + json_object_start(&w); + json_kv_string(&w, "id", id); + json_kv_string(&w, "name", name); + json_key(&w, "input"); + json_raw(&w, input_json); + json_object_end(&w); + if (ip_compose(ip, M_TOOL_REQUEST, EP_EXEC, EP_NET, b.data, + b.len) == -1 || ip_flush(ip) == -1) + errx(1, "send_req"); + buf_free(&b); +} + +/* Receive one M_TOOL_RESULT; fills content and *is_error. Returns 0/-1. */ +static int +recv_result(struct imsgproto *ip, char *content, size_t cap, int *is_error) +{ + struct fugu_msg m; + struct json_doc d; + char *c; + int t; + + if (recv_msg(ip, &m) != 1 || m.type != M_TOOL_RESULT) + return (-1); + if (json_parse(&d, m.data, m.len) == -1) { + ip_msg_free(&m); + return (-1); + } + content[0] = '\0'; + if ((t = json_obj_get(&d, 0, "content")) >= 0 && + (c = json_tok_strdup(&d, t)) != NULL) { + strlcpy(content, c, cap); + free(c); + } + t = json_obj_get(&d, 0, "is_error"); + *is_error = (t >= 0 && json_tok_streq(&d, t, "true")); + json_doc_free(&d); + ip_msg_free(&m); + return (0); +} + +static void +on_alarm(int sig) +{ + (void)sig; + dprintf(STDERR_FILENO, "exec_test: TIMED OUT\n"); + _exit(97); +} + +int +main(void) +{ + struct imsgproto ip; + char content[4096], fb[4096], td[] = "/tmp/fugu-exec.XXXXXX"; + const char *escape = "/tmp/fugu-exec-escape.txt"; + pid_t pid; + int sv[2], status, is_err; + + signal(SIGALRM, on_alarm); + signal(SIGPIPE, SIG_IGN); + alarm(60); + + if (mkdtemp(td) == NULL) + err(1, "mkdtemp"); + strlcpy(tmpdir, td, sizeof(tmpdir)); + unlink(escape); + + writefile("hello.txt", "hello world"); + writefile("dup.txt", "aXa"); /* 'a' twice -> not unique */ + + if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) + err(1, "socketpair"); + if ((pid = fork()) == -1) + err(1, "fork"); + if (pid == 0) { + struct imsgproto eip; + + close(sv[0]); + if (chdir(tmpdir) == -1) + _exit(40); + if (ip_init(&eip, sv[1]) == -1) + _exit(41); + exec_sandbox(0); /* unveil "."=tmpdir, pledge */ + exec_loop(&eip); + _exit(0); + } + close(sv[1]); + if (ip_init(&ip, sv[0]) == -1) + err(1, "ip_init"); + + /* read */ + send_req(&ip, "t1", "read", "{\"path\":\"hello.txt\"}"); + CHECK(recv_result(&ip, content, sizeof(content), &is_err) == 0, "t1 recv"); + CHECK(!is_err && strcmp(content, "hello world") == 0, + "read content '%s' err=%d", content, is_err); + + /* write */ + send_req(&ip, "t2", "write", + "{\"path\":\"new.txt\",\"content\":\"created data\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err, "write ok (%s)", content); + CHECK(readfile("new.txt", fb, sizeof(fb)) >= 0 && + strcmp(fb, "created data") == 0, "write effect '%s'", fb); + + /* edit (unique) */ + send_req(&ip, "t3", "edit", + "{\"path\":\"hello.txt\",\"old_string\":\"world\"," + "\"new_string\":\"fugu\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err, "edit ok (%s)", content); + CHECK(readfile("hello.txt", fb, sizeof(fb)) >= 0 && + strcmp(fb, "hello fugu") == 0, "edit effect '%s'", fb); + + /* edit: old_string not found */ + send_req(&ip, "t4", "edit", + "{\"path\":\"hello.txt\",\"old_string\":\"zzz\"," + "\"new_string\":\"x\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "edit not-found is error"); + + /* edit: old_string not unique */ + send_req(&ip, "t5", "edit", + "{\"path\":\"dup.txt\",\"old_string\":\"a\",\"new_string\":\"b\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "edit non-unique is error"); + + /* read missing file */ + send_req(&ip, "t6", "read", "{\"path\":\"nope.txt\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "read missing is error"); + + /* write outside the unveil'd cwd is refused */ + send_req(&ip, "t7", "write", + "{\"path\":\"/tmp/fugu-exec-escape.txt\",\"content\":\"x\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "escape write is error"); + CHECK(access(escape, F_OK) == -1, "escape file not created"); + + /* unknown tool */ + send_req(&ip, "t8", "frobnicate", "{}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "unknown tool is error"); + + /* shell: command output is captured */ + send_req(&ip, "sh1", "shell", "{\"command\":\"echo hello-shell\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "hello-shell") != NULL, + "shell echo '%s' err=%d", content, is_err); + + /* shell: a non-zero exit status is reported, not an error */ + send_req(&ip, "sh2", "shell", "{\"command\":\"exit 3\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "exit status 3") != NULL, + "shell exit code '%s' err=%d", content, is_err); + + /* grep: find a pattern with file:line output */ + writefile("src.txt", "alpha\nneedle here\nomega\n"); + send_req(&ip, "g1", "grep", + "{\"pattern\":\"needle\",\"path\":\"src.txt\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "needle") != NULL, + "grep match '%s' err=%d", content, is_err); + + /* grep: no match exits 1 but is not a tool error */ + send_req(&ip, "g2", "grep", + "{\"pattern\":\"zzznomatch\",\"path\":\"src.txt\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err, "grep no-match not an error (%s)", content); + + /* ls: lists the project directory */ + send_req(&ip, "l1", "ls", "{\"path\":\".\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "src.txt") != NULL, + "ls lists src.txt '%s' err=%d", content, is_err); + + /* find: locate a file by -name glob */ + send_req(&ip, "f1", "find", "{\"path\":\".\",\"name\":\"src.txt\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "src.txt") != NULL, + "find by name '%s' err=%d", content, is_err); + + /* allow_write gate */ + { + struct buf cfg; + + buf_init(&cfg); + buf_puts_cstr(&cfg, "{\"allow_write\":false}"); + if (ip_compose(&ip, M_CONFIG, EP_EXEC, EP_PARENT, cfg.data, + cfg.len) == -1 || ip_flush(&ip) == -1) + errx(1, "config"); + buf_free(&cfg); + } + send_req(&ip, "t9", "write", + "{\"path\":\"blocked.txt\",\"content\":\"nope\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "write blocked when allow_write=false"); + CHECK(readfile("blocked.txt", fb, sizeof(fb)) == -1, + "blocked file not created"); + + if (ip_compose(&ip, M_SHUTDOWN, EP_EXEC, EP_PARENT, NULL, 0) != -1) + ip_flush(&ip); + + if (waitpid(pid, &status, 0) == -1) + err(1, "waitpid"); + CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0, + "exec worker exit %d", status); + + /* cleanup */ + unlink(escape); + { + const char *f[] = { + "hello.txt", "dup.txt", "new.txt", "src.txt" + }; + char p[512]; + size_t i; + + for (i = 0; i < sizeof(f) / sizeof(f[0]); i++) { + snprintf(p, sizeof(p), "%s/%s", tmpdir, f[i]); + unlink(p); + } + rmdir(tmpdir); + } + + printf("exec_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + fae966d10a6b766a5029120db5cd9189ea0b4215 (mode 644) --- /dev/null +++ regress/fetch/Makefile @@ -0,0 +1,29 @@ +# Integration test for the fetch web worker (fetch_run.c, kagi.c, +# html2text.c) against a local libtls stub server. The url and httpacl +# helpers were inlined into fetch_run.c and are exercised through it; the +# tlsconn helper lives in src/common/tlsconn.c. + +PROG= fetch_test +SRCS= fetch_test.c fetch_run.c kagi.c html2text.c http.c \ + json.c buf.c util.c log.c imsgproto.c tlsconn.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +FETCHDIR= ${.CURDIR}/../../src/fetch +CFLAGS+= -I${COMMONDIR} -I${FETCHDIR} +.PATH: ${COMMONDIR} ${FETCHDIR} + +LDADD+= -ltls -lutil +DPADD+= ${LIBTLS} ${LIBUTIL} + +REGRESS_TARGETS= run-fetch_test + +run-fetch_test: ${PROG} + @td=`mktemp -d /tmp/fugu-fetch-cert.XXXXXX`; \ + openssl req -x509 -newkey rsa:2048 -nodes \ + -keyout $$td/key.pem -out $$td/cert.pem \ + -days 1 -subj '/CN=localhost' >/dev/null 2>&1; \ + ./${PROG} $$td/cert.pem $$td/key.pem; rc=$$?; \ + rm -rf $$td; exit $$rc + +.include blob - /dev/null blob + 2e8fe2cad202c565c37a9709f79a256dffe9fce6 (mode 644) --- /dev/null +++ regress/fetch/fetch_test.c @@ -0,0 +1,743 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* + * Drives the fetch worker over imsg with a real libtls stub server: a forked, + * sandboxed fetch_loop is pointed (via M_CONFIG) at a local TLS server that + * answers the Kagi search path with canned JSON -- but only when the request + * carries the expected "Authorization: Bearer " header, otherwise 401 -- and + * any other path with an HTML page. Verifies web_search formatting and that the + * key is sent (a dropped header fails the search), that the missing-token path + * errors without a connection, web_fetch HTML->text, bad-URL rejection, and that + * http_request enforces the operator's host allow/block lists. + */ + +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include + +#include + +#include "buf.h" +#include "fetch_run.h" +#include "imsgproto.h" +#include "json.h" + +static int checks, failures; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +/* A Kagi Search API v1 success envelope: results under data.search[]. */ +static const char KAGI_JSON[] = + "{\"meta\":{\"node\":\"test\"},\"data\":{" + "\"related_search\":[{\"url\":\"/search?q=x\",\"title\":\"related-thing\"}]," + "\"search\":[" + "{\"url\":\"https://example.com/a\",\"title\":\"Example A\"," + "\"snippet\":\"first snippet\"}," + "{\"url\":\"https://example.com/b\",\"title\":\"Example B\"," + "\"snippet\":\"second snippet\"}]," + "\"video\":[{\"url\":\"https://v/1\",\"title\":\"a video\"}]}}"; + +static const char HTML_PAGE[] = + "T" + "

Heading

Hello & welcome to fugu.

" + ""; + +static int +tls_write_all(struct tls *t, const void *buf, size_t len) +{ + const unsigned char *p = buf; + size_t off = 0; + ssize_t n; + + while (off < len) { + n = tls_write(t, p + off, len - off); + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) + continue; + if (n < 0) + return (-1); + off += (size_t)n; + } + return (0); +} + +static void +stub_server(int lfd, const char *cert, const char *key) +{ + struct tls_config *cfg; + struct tls *srv; + + if ((cfg = tls_config_new()) == NULL) + _exit(21); + if (tls_config_set_cert_file(cfg, cert) == -1 || + tls_config_set_key_file(cfg, key) == -1) + _exit(22); + if ((srv = tls_server()) == NULL || tls_configure(srv, cfg) == -1) + _exit(23); + + for (;;) { + struct tls *ctx; + struct buf resp; + char req[4096]; + const char *body, *ctype, *status = "200 OK"; + size_t off = 0; + ssize_t n; + int cfd; + + if ((cfd = accept(lfd, NULL, NULL)) == -1) + break; + if (tls_accept_socket(srv, &ctx, cfd) == -1) { + close(cfd); + continue; + } + do { + n = tls_handshake(ctx); + } while (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT); + if (n == -1) { + tls_free(ctx); + close(cfd); + continue; + } + /* Read the headers, then the POST body (so we can see the query). */ + while (off < sizeof(req) - 1) { + n = tls_read(ctx, req + off, sizeof(req) - 1 - off); + if (n == TLS_WANT_POLLIN || n == TLS_WANT_POLLOUT) + continue; + if (n <= 0) + break; + off += (size_t)n; + req[off] = '\0'; + if (strstr(req, "\r\n\r\n") != NULL) + break; + } + { + char *cl = strstr(req, "Content-Length:"); + char *end = strstr(req, "\r\n\r\n"); + size_t need, have; + + if (cl != NULL && end != NULL) { + need = (size_t)strtol(cl + 15, NULL, 10); + have = off - (size_t)(end + 4 - req); + while (have < need && off < sizeof(req) - 1) { + n = tls_read(ctx, req + off, + sizeof(req) - 1 - off); + if (n == TLS_WANT_POLLIN || + n == TLS_WANT_POLLOUT) + continue; + if (n <= 0) + break; + off += (size_t)n; + have += (size_t)n; + } + } + } + req[off] = '\0'; + + /* + * The /echo endpoint reports back the received body length and a + * hex dump of its first bytes, so the test can prove a body with + * an embedded NUL byte (e.g. "ab\0cd") survived end-to-end + * rather than being truncated at the first NUL by a strlen-based + * writer. Used only by the embedded-NUL regress test. + */ + if (strstr(req, "POST /echo") != NULL) { + char *end = strstr(req, "\r\n\r\n"); + size_t blen, i; + static char echobody[256]; + + blen = (end != NULL) ? + off - (size_t)(end + 4 - req) : 0; + snprintf(echobody, sizeof(echobody), + "body_len=%zu hex=", blen); + for (i = 0; i < blen && i < 16; i++) { + char hex[4]; + snprintf(hex, sizeof(hex), "%02x ", + (unsigned char)(end + 4)[i]); + strlcat(echobody, hex, sizeof(echobody)); + } + body = echobody; + ctype = "text/plain"; + } else if (strstr(req, "POST /api/v1/search") != NULL) { + ctype = "application/json"; + /* + * Authenticate like the real Kagi API: require the + * "Authorization: Bearer " header carrying the token + * the parent delivered. Without it the search fails, so + * the success path asserts the key is sent correctly. + */ + if (strstr(req, "Bearer test-token") == NULL) { + status = "401 Unauthorized"; + body = "{\"errors\":[{\"code\":\"unauthorized\"," + "\"message\":\"missing or bad token\"}]}"; + } else if (strstr(req, "boom") != NULL) { + /* A non-200 with a v1 error envelope. */ + status = "402 Payment Required"; + body = "{\"errors\":[{\"code\":\"out_of_funds\"," + "\"message\":\"Out of credits\"}]}"; + } else { + body = KAGI_JSON; + } + } else { + body = HTML_PAGE; + ctype = "text/html; charset=utf-8"; + } + buf_init(&resp); + buf_printf(&resp, "HTTP/1.1 %s\r\nContent-Type: %s\r\n" + "Content-Length: %zu\r\nConnection: close\r\n\r\n", + status, ctype, strlen(body)); + buf_puts_cstr(&resp, body); + (void)tls_write_all(ctx, resp.data, resp.len); + buf_free(&resp); + tls_close(ctx); + tls_free(ctx); + close(cfd); + } + tls_free(srv); + tls_config_free(cfg); + _exit(0); +} + +static int +recv_msg(struct imsgproto *ip, struct fugu_msg *m) +{ + int r; + + for (;;) { + if ((r = ip_get(ip, m)) == 1) + return (1); + if (r == -1) + return (-1); + if (ip_read(ip) <= 0) + return (0); + } +} + +static void +send_msg(struct imsgproto *ip, uint32_t type, const char *json) +{ + if (ip_compose(ip, type, EP_FETCH, EP_NET, json, + json != NULL ? strlen(json) : 0) == -1 || ip_flush(ip) == -1) + errx(1, "send_msg"); +} + +static void +send_req(struct imsgproto *ip, const char *id, const char *name, + const char *input_json) +{ + struct json_writer w; + struct buf b; + + buf_init(&b); + json_writer_init(&w, &b); + json_object_start(&w); + json_kv_string(&w, "id", id); + json_kv_string(&w, "name", name); + json_key(&w, "input"); + json_raw(&w, input_json); + json_object_end(&w); + if (ip_compose(ip, M_TOOL_REQUEST, EP_FETCH, EP_NET, b.data, + b.len) == -1 || ip_flush(ip) == -1) + errx(1, "send_req"); + buf_free(&b); +} + +static int +recv_result(struct imsgproto *ip, char *content, size_t cap, int *is_error) +{ + struct fugu_msg m; + struct json_doc d; + char *c; + int t; + + if (recv_msg(ip, &m) != 1 || m.type != M_TOOL_RESULT) + return (-1); + if (json_parse(&d, m.data, m.len) == -1) { + ip_msg_free(&m); + return (-1); + } + content[0] = '\0'; + if ((t = json_obj_get(&d, 0, "content")) >= 0 && + (c = json_tok_strdup(&d, t)) != NULL) { + strlcpy(content, c, cap); + free(c); + } + t = json_obj_get(&d, 0, "is_error"); + *is_error = (t >= 0 && json_tok_streq(&d, t, "true")); + json_doc_free(&d); + ip_msg_free(&m); + return (0); +} + +static void +on_alarm(int sig) +{ + (void)sig; + dprintf(STDERR_FILENO, "fetch_test: TIMED OUT\n"); + _exit(97); +} + +int +main(int argc, char *argv[]) +{ + struct imsgproto ip; + struct sockaddr_in sa; + struct fugu_msg m; + socklen_t slen; + char content[8192], cfg[1024], url[256]; + pid_t sp, fp; + int lfd, one = 1, port, sv[2], status, is_err; + + if (argc != 3) + errx(1, "usage: %s cert.pem key.pem", argv[0]); + + signal(SIGALRM, on_alarm); + signal(SIGPIPE, SIG_IGN); + alarm(90); + + if ((lfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) + err(1, "socket"); + setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)); + memset(&sa, 0, sizeof(sa)); + sa.sin_family = AF_INET; + sa.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + if (bind(lfd, (struct sockaddr *)&sa, sizeof(sa)) == -1) + err(1, "bind"); + if (listen(lfd, 4) == -1) + err(1, "listen"); + slen = sizeof(sa); + if (getsockname(lfd, (struct sockaddr *)&sa, &slen) == -1) + err(1, "getsockname"); + port = ntohs(sa.sin_port); + + if ((sp = fork()) == -1) + err(1, "fork stub"); + if (sp == 0) { + stub_server(lfd, argv[1], argv[2]); + _exit(0); + } + close(lfd); + + if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == -1) + err(1, "socketpair"); + if ((fp = fork()) == -1) + err(1, "fork fetch"); + if (fp == 0) { + struct imsgproto fip; + + close(sv[0]); + if (ip_init(&fip, sv[1]) == -1) + _exit(41); + fetch_loop(&fip); + _exit(0); + } + close(sv[1]); + if (ip_init(&ip, sv[0]) == -1) + err(1, "ip_init"); + + /* fetch announces itself, then we point it at the stub. */ + CHECK(recv_msg(&ip, &m) == 1 && m.type == M_READY, "expect M_READY"); + ip_msg_free(&m); + + /* allow_private: the stub is on loopback, which the SSRF guard blocks. */ + snprintf(cfg, sizeof(cfg), "{\"kagi_host\":\"localhost\"," + "\"kagi_port\":\"%d\",\"cafile\":\"%s\",\"allow_private\":true}", + port, argv[1]); + send_msg(&ip, M_CONFIG, cfg); + + /* web_search before the token arrives: error, and no connection made. */ + send_req(&ip, "s0", "web_search", "{\"query\":\"anything\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "token") != NULL, + "web_search w/o token errors: '%s'", content); + + send_msg(&ip, M_SECRET, "test-token"); + + /* + * web_search with the token: the stub returns results only when the + * request carried "Authorization: Bearer test-token", so a successful, + * formatted result here also proves the Kagi key is sent correctly. + */ + send_req(&ip, "s1", "web_search", "{\"query\":\"fugu shell\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err, "web_search ok / Bearer auth header accepted (%s)", content); + CHECK(strstr(content, "Example A") != NULL && + strstr(content, "https://example.com/a") != NULL && + strstr(content, "first snippet") != NULL, + "web_search result A present: '%s'", content); + CHECK(strstr(content, "Example B") != NULL, + "web_search result B present"); + CHECK(strstr(content, "related-thing") == NULL, + "web_search drops related (t=1) entries"); + + /* A non-200 Kagi reply surfaces its error message, not just the code. */ + send_req(&ip, "s2", "web_search", "{\"query\":\"boom\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "402") != NULL && + strstr(content, "Out of credits") != NULL, + "web_search surfaces Kagi's error message: '%s'", content); + + /* web_fetch: HTML reduced to text, scripts/styles dropped. */ + snprintf(url, sizeof(url), "https://localhost:%d/page", port); + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"%s\"}", url); + buf_putc(&in, '\0'); + send_req(&ip, "f1", "web_fetch", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err, "web_fetch ok (%s)", content); + CHECK(strstr(content, "Heading") != NULL && + strstr(content, "Hello & welcome to fugu") != NULL, + "web_fetch text content: '%s'", content); + CHECK(strstr(content, "alert") == NULL && + strstr(content, "color") == NULL, + "web_fetch drops script/style: '%s'", content); + + /* web_fetch with an unsupported scheme: rejected before any connect. */ + send_req(&ip, "f2", "web_fetch", "{\"url\":\"ftp://localhost/x\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "web_fetch bad URL is error"); + + /* http_request: enforce the operator's host allow/block lists. */ + send_msg(&ip, M_CONFIG, + "{\"http_allow\":\"localhost\",\"http_block\":\"localhost/secret\"}"); + + /* Allowed host: reaches the stub; raw body returned with a status line. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/page\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h1", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP 200") != NULL, + "http_request allowed host ok: '%s'", content); + CHECK(strstr(content, "Heading") != NULL && strstr(content, "alert") != NULL, + "http_request returns the RAW body (no html2text): '%s'", content); + + /* Host not in the allowlist: refused by the gate, before any connection. */ + send_req(&ip, "h2", "http_request", "{\"url\":\"https://evil.example/x\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "not allowed") != NULL, + "http_request denies a non-allowlisted host: '%s'", content); + + /* Blocked path on an allowed host: the blocklist wins. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/secret/data\"}", + port); + buf_putc(&in, '\0'); + send_req(&ip, "h3", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "not allowed") != NULL, + "http_request blocklist denies a blocked path: '%s'", content); + + /* Non-https URL: rejected. */ + send_req(&ip, "h4", "http_request", "{\"url\":\"http://localhost/x\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err, "http_request rejects a non-https URL"); + + /* + * Embedded-NUL body: the JSON body decodes to a real NUL byte. + * Before the fix, do_http_request derived Content-Length from + * strlen(reqbody) and truncated the body at the first NUL (so the + * stub would see body_len=2, hex=68 69). After the fix it must + * survive: body_len=5, hex=68 69 00 21 21. The stub's /echo path + * responds with a plain-text trace of what it actually received. + */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/echo\"," + "\"method\":\"POST\"," + "\"body\":\"hi\\u0000!!\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "n0", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err, "http_request with NUL-bearing body succeeds: '%s'", + content); + CHECK(strstr(content, "body_len=5") != NULL, + "http_request body NOT truncated at NUL: '%s'", content); + CHECK(strstr(content, "68 69 00 21 21") != NULL, + "http_request body bytes intact incl. NUL: '%s'", content); + + /* URL with an embedded NUL: rejected before any connect. */ + send_req(&ip, "n1", "http_request", + "{\"url\":\"https://localhost\\u0000evil/x\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "NUL") != NULL, + "http_request rejects URL with embedded NUL: '%s'", content); + + /* Header value with an embedded NUL: rejected, no request sent. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/echo\"," + "\"method\":\"POST\"," + "\"headers\":{\"x-trace\":\"a\\u0000b\"}," + "\"body\":\"x\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "n2", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "NUL") != NULL, + "http_request rejects header value with NUL: '%s'", content); + + /* web_search query with an embedded NUL: rejected, no Kagi call. */ + send_req(&ip, "n3", "web_search", + "{\"query\":\"foo\\u0000bar\"}"); + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "NUL") != NULL, + "web_search rejects query with NUL: '%s'", content); + + /* + * Path-prefix ACL boundary: an allowlist of "localhost/v1" must not let + * a sibling path like "/v1admin/secrets" slip through on a raw prefix + * match -- the prefix has to end at a path-segment boundary ('/' or end + * of path). Legitimate paths ("/v1", "/v1/", "/v1/whatever") are still + * allowed. + */ + send_msg(&ip, M_CONFIG, + "{\"http_allow\":\"localhost/v1\",\"http_block\":\"\"}"); + + /* BAD path: must be REJECTED (regression for the boundary bypass). */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, + "{\"url\":\"https://localhost:%d/v1admin/secrets\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h5", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "not allowed") != NULL, + "http_request path-prefix boundary: /v1admin/secrets is rejected " + "by an allowlist of /v1: '%s'", content); + + /* GOOD: bare "/v1" still allowed. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/v1\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h6", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request path-prefix allows /v1 itself: '%s'", content); + + /* GOOD: "/v1/" (trailing slash) still allowed. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/v1/\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h7", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request path-prefix allows /v1/ itself: '%s'", content); + + /* GOOD: "/v1/whatever" (deeper path under the prefix) still allowed. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, + "{\"url\":\"https://localhost:%d/v1/whatever\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h8", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request path-prefix allows /v1/whatever: '%s'", content); + + /* + * GOOD: "/v1?token=abc" -- a query string sits at the prefix boundary, + * just like '/'. Before the boundary set was extended, '?' was treated + * as a sibling-path byte and this URL was wrongly denied. + */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, + "{\"url\":\"https://localhost:%d/v1?token=abc\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h9", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request path-prefix allows /v1?token=abc: '%s'", content); + + /* + * GOOD: "/v1#frag" -- a fragment also marks the prefix boundary. + * In practice url_parse() already drops '#...' before the ACL sees + * the path, so this case asserts the end-to-end behaviour rather than + * exercising the boundary byte directly. + */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, + "{\"url\":\"https://localhost:%d/v1#frag\"}", port); + buf_putc(&in, '\0'); + send_req(&ip, "h10", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request path-prefix allows /v1#frag: '%s'", content); + + /* + * Host-match ACL boundary: with an EXACT pattern "localhost", a host + * like "evillocalhost" must not match by string-contains; the rule is + * full-equality on non-dot patterns, suffix match only when the + * pattern starts with '.'. These check both halves of the rule -- a + * regression would let evilhosts.example.com slip past an allowlist of + * "example.com". Detected as the ACL "not allowed" rejection (which + * fires before dial), so the lookup of the bad host never happens. + */ + send_msg(&ip, M_CONFIG, + "{\"http_allow\":\"localhost\",\"http_block\":\"\"}"); + + /* BAD host: "evillocalhost" must NOT match the exact pattern "localhost". */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, + "{\"url\":\"https://evillocalhost:%d/v1\"}", port); + buf_terminate(&in); + send_req(&ip, "h11", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "not allowed") != NULL, + "http_request host-exact rejects evillocalhost vs localhost: '%s'", + content); + + /* GOOD host: "localhost" matches itself exactly. */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/v1\"}", port); + buf_terminate(&in); + send_req(&ip, "h12", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request host-exact allows localhost: '%s'", content); + + /* + * Dot-suffix rule: ".localhost" matches the bare "localhost" AND any + * "*.localhost", but NOT "evillocalhost" -- the leading '.' anchors + * the suffix at a label boundary, so a pattern like ".example.com" + * cannot be slipped by "evilexample.com". We exercise both the + * bare-domain branch (host equals pattern minus the dot) and the + * boundary rejection. The other branch (real "*.localhost") would + * need a multi-SAN cert; the boundary case below is what catches the + * historical bypass shape. + */ + send_msg(&ip, M_CONFIG, + "{\"http_allow\":\".localhost\",\"http_block\":\"\"}"); + + /* GOOD host: bare "localhost" matches pattern ".localhost". */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, "{\"url\":\"https://localhost:%d/v1\"}", port); + buf_terminate(&in); + send_req(&ip, "h13", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(!is_err && strstr(content, "HTTP") != NULL, + "http_request dot-suffix allows bare localhost vs .localhost: '%s'", + content); + + /* BAD host: "evillocalhost" must NOT match dot-suffix ".localhost". */ + { + struct buf in; + + buf_init(&in); + buf_printf(&in, + "{\"url\":\"https://evillocalhost:%d/v1\"}", port); + buf_terminate(&in); + send_req(&ip, "h14", "http_request", in.data); + buf_free(&in); + } + recv_result(&ip, content, sizeof(content), &is_err); + CHECK(is_err && strstr(content, "not allowed") != NULL, + "http_request dot-suffix rejects evillocalhost vs .localhost " + "(label-boundary anchor): '%s'", content); + + send_msg(&ip, M_SHUTDOWN, NULL); + if (waitpid(fp, &status, 0) == -1) + err(1, "waitpid fetch"); + CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0, + "fetch worker exit %d", status); + + kill(sp, SIGTERM); + waitpid(sp, &status, 0); + + printf("fetch_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + 8dd94e5c75ff95ca3a535ca4cf4d8b4422654569 (mode 644) --- /dev/null +++ regress/fuzzy/Makefile @@ -0,0 +1,16 @@ +# Unit tests for the fuzzy matcher (src/common/fuzzy.c). + +PROG= fuzzy_test +SRCS= fuzzy_test.c fuzzy.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +REGRESS_TARGETS= run-fuzzy_test + +run-fuzzy_test: ${PROG} + ./${PROG} + +.include blob - /dev/null blob + 6af9917ae56270df7a40241160b02ff68e1a8df7 (mode 644) --- /dev/null +++ regress/fuzzy/fuzzy_test.c @@ -0,0 +1,90 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include + +#include "fuzzy.h" + +static int checks; +static int failures; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +static void +test_basic(void) +{ + int s; + + CHECK(fuzzy_match("", "anything", &s) == 1 && s == 0, + "empty needle matches everything, score 0"); + CHECK(fuzzy_match("son", "claude-sonnet-4-6", &s) == 1, + "subsequence matches"); + CHECK(fuzzy_match("gpt4", "gpt-4o-mini", &s) == 1, + "scattered subsequence matches (g p t 4)"); + CHECK(fuzzy_match("xyz", "claude-sonnet", &s) == 0 && s == 0, + "no subsequence -> no match, score 0"); + CHECK(fuzzy_match("sonnetx", "claude-sonnet", &s) == 0, + "needle longer than available tail -> no match"); + CHECK(fuzzy_match("SON", "claude-sonnet", &s) == 1, + "case-insensitive (upper needle)"); + CHECK(fuzzy_match("son", "CLAUDE-SONNET", &s) == 1, + "case-insensitive (upper haystack)"); +} + +static void +test_scoring(void) +{ + int a, b; + + /* A boundary-aligned hit outscores a mid-word one. */ + CHECK(fuzzy_match("son", "claude-sonnet", &a) == 1 && + fuzzy_match("son", "personals", &b) == 1 && a > b, + "boundary-aligned 'son' (a=%d) beats mid-word (b=%d)", a, b); + + /* A consecutive run outscores the same chars scattered mid-word. */ + CHECK(fuzzy_match("gpt", "gpt-4o", &a) == 1 && + fuzzy_match("gpt", "gxpxtx", &b) == 1 && a > b, + "consecutive 'gpt' (a=%d) beats scattered mid-word (b=%d)", a, b); + + /* Matching each segment start (acronym-like) also scores well. */ + CHECK(fuzzy_match("gpt", "g-p-t-x", &a) == 1 && a > 12, + "boundary-aligned acronym 'gpt' scores high (a=%d)", a); + + /* A full prefix is a strong score (boundary + run). */ + CHECK(fuzzy_match("claude", "claude-sonnet-4-6", &a) == 1 && a > 20, + "prefix 'claude' scores high (a=%d)", a); + + /* NULL score pointer is tolerated. */ + CHECK(fuzzy_match("son", "claude-sonnet", NULL) == 1, + "NULL score pointer ok"); +} + +int +main(void) +{ + test_basic(); + test_scoring(); + printf("fuzzy_test: %d checks, %d failures\n", checks, failures); + return (failures ? 1 : 0); +} blob - /dev/null blob + 740d8b5359f0a873e9531869ba1f8096ee7fa7fc (mode 644) --- /dev/null +++ regress/html2text/Makefile @@ -0,0 +1,19 @@ +# Unit test for the HTML-to-text reducer (src/common/html2text.c). + +PROG= html2text_test +SRCS= html2text_test.c html2text.c buf.c util.c log.c +NOMAN= yes + +COMMONDIR= ${.CURDIR}/../../src/common +CFLAGS+= -I${COMMONDIR} +.PATH: ${COMMONDIR} + +LDADD+= -lutil +DPADD+= ${LIBUTIL} + +REGRESS_TARGETS= run-html2text_test + +run-html2text_test: ${PROG} + ./${PROG} + +.include blob - /dev/null blob + 6a0f6a9bcc0791688503c5e0ffa08803d42b4c84 (mode 644) --- /dev/null +++ regress/html2text/html2text_test.c @@ -0,0 +1,299 @@ +/* + * Copyright (c) 2026 Isaac + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include +#include +#include + +#include "buf.h" +#include "html2text.h" + +static int checks, failures; + +#define CHECK(cond, ...) do { \ + checks++; \ + if (!(cond)) { \ + failures++; \ + fprintf(stderr, "FAIL %s:%d: ", __func__, __LINE__); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, "\n"); \ + } \ +} while (0) + +/* Run html2text and compare the (NUL-terminated) result exactly. */ +static void +eq(const char *html, const char *want) +{ + struct buf b; + + buf_init(&b); + html2text(html, strlen(html), &b); + buf_putc(&b, '\0'); + CHECK(strcmp(b.data, want) == 0, "in=<%s> got=<%s> want=<%s>", + html, b.data, want); + buf_free(&b); +} + +/* Run html2text and check a substring is present / absent. */ +static void +has(const char *html, const char *needle, int present) +{ + struct buf b; + int found; + + buf_init(&b); + html2text(html, strlen(html), &b); + buf_putc(&b, '\0'); + found = (strstr(b.data, needle) != NULL); + CHECK(found == present, "in=<%s> expected %s'%s' in <%s>", html, + present ? "" : "no ", needle, b.data); + buf_free(&b); +} + +/* + * Validate that bytes[0..n) is a well-formed UTF-8 sequence (no truncated + * multi-byte codepoints, no stray continuation bytes, no overlongs). + */ +static int +is_well_formed_utf8(const unsigned char *bytes, size_t n) +{ + size_t i = 0; + + while (i < n) { + unsigned char c = bytes[i]; + size_t need, k; + unsigned long cp; + + if (c < 0x80) { + i++; + continue; + } + if ((c & 0xe0) == 0xc0) { + need = 2; + cp = c & 0x1f; + } else if ((c & 0xf0) == 0xe0) { + need = 3; + cp = c & 0x0f; + } else if ((c & 0xf8) == 0xf0) { + need = 4; + cp = c & 0x07; + } else + return (0); /* stray continuation or invalid lead */ + if (i + need > n) + return (0); /* truncated sequence */ + for (k = 1; k < need; k++) { + if ((bytes[i + k] & 0xc0) != 0x80) + return (0); + cp = (cp << 6) | (bytes[i + k] & 0x3f); + } + /* reject overlongs and surrogates / out-of-range */ + if (need == 2 && cp < 0x80) + return (0); + if (need == 3 && cp < 0x800) + return (0); + if (need == 4 && cp < 0x10000) + return (0); + if (cp >= 0xd800 && cp <= 0xdfff) + return (0); + if (cp > 0x10ffff) + return (0); + i += need; + } + return (1); +} + +/* + * Regress for the low-UTF-8 boundary bug: drive html2text right up to + * HTML2TEXT_MAXOUT and then emit a 3-byte UTF-8 codepoint. Before the fix the + * cap was checked one byte at a time, so the lead byte slipped through and the + * continuation bytes were dropped, leaving a malformed sequence at the tail. + * After the fix the whole codepoint is dropped and the output stays + * well-formed UTF-8 of length <= HTML2TEXT_MAXOUT. + */ +static void +test_utf8_boundary(void) +{ + const char *emdash = "—"; /* U+2014, 3 bytes UTF-8 */ + size_t emlen = strlen(emdash); + size_t fill, total, off; + char *html; + struct buf b; + + /* + * Fill so the first emitted byte of the em dash lands at offset + * HTML2TEXT_MAXOUT - 1. With the old code byte 1 fits, bytes 2-3 are + * dropped, and the buffer ends in a malformed UTF-8 lead byte. + */ + fill = HTML2TEXT_MAXOUT - 1; + total = fill + emlen + 1; + html = malloc(total); + CHECK(html != NULL, "malloc(%zu) failed", total); + if (html == NULL) + return; + memset(html, 'a', fill); + memcpy(html + fill, emdash, emlen); + html[total - 1] = '\0'; + + buf_init(&b); + html2text(html, fill + emlen, &b); + CHECK(b.len <= HTML2TEXT_MAXOUT, "len %zu exceeds cap %u", b.len, + HTML2TEXT_MAXOUT); + CHECK(is_well_formed_utf8((const unsigned char *)b.data, b.len), + "output not well-formed UTF-8 (len=%zu, last byte=0x%02x)", + b.len, b.len > 0 ? + (unsigned char)b.data[b.len - 1] : 0); + /* + * Also try a 2-byte (U+00A9 copyright) and a 4-byte (U+1F600) codepoint + * at the same boundary to exercise every encoding width. + */ + buf_free(&b); + buf_init(&b); + memcpy(html + fill, "©", 6); + html2text(html, fill + 6, &b); + CHECK(b.len <= HTML2TEXT_MAXOUT, "len %zu exceeds cap (2-byte)", + b.len); + CHECK(is_well_formed_utf8((const unsigned char *)b.data, b.len), + "2-byte boundary: output not well-formed UTF-8"); + + buf_free(&b); + buf_init(&b); + memcpy(html + fill, "😀", 9); + html2text(html, fill + 9, &b); + CHECK(b.len <= HTML2TEXT_MAXOUT, "len %zu exceeds cap (4-byte)", + b.len); + CHECK(is_well_formed_utf8((const unsigned char *)b.data, b.len), + "4-byte boundary: output not well-formed UTF-8"); + + /* + * And a sweep across a few offsets just before the cap so we hit the + * case where the first byte fits but the rest do not, regardless of + * exactly which offset the writes started from. + */ + for (off = 2; off <= 5; off++) { + fill = HTML2TEXT_MAXOUT - off; + memset(html, 'a', fill); + memcpy(html + fill, emdash, emlen); + buf_free(&b); + buf_init(&b); + html2text(html, fill + emlen, &b); + CHECK(b.len <= HTML2TEXT_MAXOUT, + "sweep off=%zu: len %zu exceeds cap", off, b.len); + CHECK(is_well_formed_utf8((const unsigned char *)b.data, + b.len), + "sweep off=%zu: output not well-formed UTF-8", off); + } + buf_free(&b); + free(html); +} + +/* + * Regress for the named-entity boundary bug: emit_named used to write its + * replacement one byte at a time, so a 3-byte expansion like "..." for … + * could be cut mid-sequence at HTML2TEXT_MAXOUT, leaving a stray "." or ".." + * that is logically half a replacement. After the fix the whole replacement + * is dropped when it doesn't fit, so the output never ends in a fragment. + */ +static void +test_named_boundary(void) +{ + const char *hellip = "…"; /* expands to "..." (3 bytes) */ + size_t hlen = strlen(hellip); + size_t fill, total; + char *html; + struct buf b; + + /* + * Fill so the output is at the cap minus 2 bytes; the 3-byte "..." + * replacement does not fit. Before the fix the first two '.' bytes + * slipped through and the third was dropped, leaving a stray ".." at + * the tail. After the fix the whole "..." is dropped. + */ + fill = HTML2TEXT_MAXOUT - 2; + total = fill + hlen + 1; + html = malloc(total); + CHECK(html != NULL, "malloc(%zu) failed", total); + if (html == NULL) + return; + memset(html, 'a', fill); + memcpy(html + fill, hellip, hlen); + html[total - 1] = '\0'; + + buf_init(&b); + html2text(html, fill + hlen, &b); + CHECK(b.len <= HTML2TEXT_MAXOUT, "len %zu exceeds cap %u", b.len, + HTML2TEXT_MAXOUT); + /* + * The replacement must be all-or-nothing: a trailing '.' would mean a + * fragment of "..." leaked through. The fill bytes are 'a', so any + * trailing '.' can only have come from the truncated replacement. + */ + CHECK(b.len == 0 || b.data[b.len - 1] != '.', + "output ends in stray '.' (len=%zu)", b.len); + buf_free(&b); + free(html); +} + +int +main(void) +{ + eq("

Hello world

", "Hello world"); + eq("a&b", "a&b"); + eq("xAy", "xAy"); + eq("xAy", "xAy"); + eq("a b", "a b"); + eq(" spaced out ", "spaced out"); + eq("
a
b
", "a\n\nb"); + eq("&foo;", "&foo;"); /* unknown entity kept */ + eq("plain text", "plain text"); + eq("link", "link"); /* attribute scanning */ + eq("a
b", "a\nb"); + + /* script / style / comment content is dropped */ + has("hi", "alert", 0); + has("hi", "hi", 1); + has("Y", "color", 0); + has("Y", "Y", 1); + has("visible", "secret", 0); + has("visible", "visible", 1); + + /* a tag with '>' inside a quoted attribute is one tag */ + has("\"ab\">text", "text", 1); + has("\"ab\">text", "a>b", 0); + + /* entity that decodes to a multibyte codepoint stays intact */ + has("—", "\xe2\x80\x94", 1); /* em dash U+2014 */ + + /* a fake close tag (prefix without delimiter) must NOT end the skip */ + has("VISIBLE", "LEAK", 0); + has("VISIBLE", "HIDDEN", 0); + has("VISIBLE", "VISIBLE", 1); + /* a real close with trailing space/attrs still ends the skip */ + eq("after", "after"); + eq("