commit - 32a9a63b07ec43fcd0d3100fadfcea52fab6ed0c
commit + f4e7405e309a640148eef4c8a266ce94c4a23b6d
blob - 49ea1c5d7760eab9c7240161e93c3ba7afa37c7f
blob + 4a2800dd74c9fbf66a7cd527cbb7d4dc8e462aa5
--- CONTEXT.md
+++ CONTEXT.md
`libexec/fugu/`: **fugu-tty** (front end, terminal), **fugu-api**
(model-credential custodian), **fugu-web** (search custodian and
hostile-content blast room), **fugu-tool** (tool executor; read-only
-role instance serves subagents), **fugu-editor** (compose spawn room).
+role instance serves subagents), **fugu-artifact** (Owner-private retained
+Tool output, quota, and bounded retrieval), **fugu-editor** (compose spawn
+room).
**Protect list**:
Owner-configured paths unveiled away from every tool-executing
**Tool**:
A capability offered to the model (read, write, edit, multi_edit,
-shell, shell_bg/output/kill, grep, find, ls, web_search, web_fetch,
-http_request, skill, agent).
+shell, shell_bg/output/kill, output_read, grep, find, ls, web_search,
+web_fetch, http_request, skill, agent).
**Subagent**:
An ephemeral, kernel-enforced read-only conversation spawned by the
## Relationships
-- The **Coordinator** alone talks to every worker; **Workers** never
- talk to each other (authority map, I10).
+- The **Coordinator** alone exchanges control messages with every worker.
+ **Workers** have no direct imsg channel. The sole direct data plane is one
+ correlated, bounded pipe protocol from the persistent Lead **fugu-tool** to
+ **fugu-artifact**, with its producer endpoint checked and relayed by the
+ Coordinator; a Subagent Role never receives one (authority map, I10).
+ OpenBSD's bidirectional `pipe(2)` extension reports both endpoints as
+ `O_RDWR`, so the one-way flow is enforced by Role and protocol ownership,
+ not by claiming a kernel write-only access mode.
- The **Owner** is fugu's only human user. The **Invoking user** is that
person's OS identity for filesystem ownership and legacy match policy, not a
second domain participant or configurable subscription identity.
- **Effort** applies unchanged to every Lead Generation and survives Provider
and model selection. A Subagent receives its inherited or explicit value at
launch and retains it for every child Generation.
+- A committed **Artifact** belongs to its **Session** and survives resume,
+ **Compaction**, and clear boundaries until quota eviction. The Journal owns
+ the structured id/size record; **fugu-artifact** alone owns raw storage, and
+ `output_read` returns only a bounded id-addressed page.
## Example dialogue
blob - c43906d6ddbf0e54a107b625cd3db44f786c9a02
blob + cf3089f13ac129c627887206cc546b0d709bc8da
--- docs/adr/0001-process-topology.md
+++ docs/adr/0001-process-topology.md
-# Coordinator plus five single-purpose workers
+# Coordinator plus six single-purpose workers
The security invariants (handoff/invariants.md I1–I15) force several
separations but leave process count and topology free. We chose a
exec after the CA bundle loads), **fugu-web** (search-credential
custodian; fetch, HTML-to-text, SSRF guard), **fugu-tool** (tool
executor; project tree unveiled; the same binary runs in a read-only
-role for subagents, I15), and **fugu-editor** (minimal spawn room for
-the external-editor compose key).
+role for subagents, I15), **fugu-artifact** (Owner-private raw Tool-output
+storage, quota accounting, and bounded page retrieval), and **fugu-editor**
+(minimal spawn room for the external-editor compose key). The
+Artifact Role alone has `rwc` access to the managed Artifact root. It has no
+project, configuration, Journal, network, tty, credential, or exec authority.
+The Coordinator remains the only Journal writer and retains no Artifact path
+promise.
## Considered options
- *Maximal separation (dedicated journal writer, one executor per
subagent):* rejected as decorative; no invariant names those walls.
"Fewer, stronger walls beat many decorative ones" (invariants.md §5).
+- *Coordinator-owned Artifact files:* rejected because the Coordinator's
+ steady pledge intentionally has no path-open promise. Restoring one would
+ combine the authority hub and private raw-output storage.
+- *Project-local Artifact files:* rejected because model-selected Tools and
+ subprocesses can read the project tree, while retained output can contain
+ secrets and must survive outside the Projection.
+- *A general storage service:* rejected as a shallow ambient capability. The
+ Artifact Role exposes only fixed Session-bound begin/settle/read/quota
+ exchanges and cannot address arbitrary paths.
## Consequences
Sharing the canonical Turn Implementation with the read-only Subagent Role
does not move either runtime or add a channel; ADR-0005 records that
refinement.
+Artifact control and terminal messages remain Coordinator-brokered. The sole
+Worker-to-Worker data-plane exception is the bounded one-way protocol over a
+pipe whose producer endpoint the Coordinator relays from `fugu-artifact` to
+the Lead `fugu-tool` for one correlated call. OpenBSD's bidirectional pipe
+extension reports both endpoints as `O_RDWR`; exact validation therefore
+requires a blocking FIFO with that access mode and `CLOEXEC`, while Role
+ownership supplies the one-way flow. Neither Role uses the endpoint in the
+reverse direction, no endpoint can become a path or general protocol, and the
+read-only Subagent Role never receives one.
blob - 5e7b0ba64fd36ebb0ad2668c447cb5c0028c7267
blob + 755d1afe65074b216dab7ac90eb8fc0a3243c868
--- docs/adr/0003-bespoke-privsep-module.md
+++ docs/adr/0003-bespoke-privsep-module.md
`getpeereid` ACLs, `chroot`, `daemon(3)`, config reload, and
per-process privilege drop to an unprivileged uid. fugu is a
foreground, single-session, per-invocation program with a
-hub-and-spoke authority map (ADR-0001) — the coordinator talks to
-each worker; workers never talk to each other.
+hub-and-spoke control authority map (ADR-0001) — the coordinator talks to
+each worker; workers never exchange control messages directly. One later
+load-bearing exception carries no control protocol: for an in-scope Lead Tool
+call, the Coordinator relays a checked pipe producer endpoint from
+`fugu-artifact` to `fugu-tool`, forming a bounded one-way raw-byte protocol.
+The Artifact Role keeps the consumer endpoint, the descriptor is correlated
+to that call, and Subagent Roles never accept it. OpenBSD exposes both pipe
+endpoints as `O_RDWR`; the Roles never exercise the reverse direction.
We therefore write a small bespoke `priv` module that copies vmd's
*discipline* — `socketpair`+`fork`+`execve` with the child's channel
terminal business, `closefrom`, `setproctitle`, and deliberate
reaping — but none of its framework: no control socket, no chroot
(unveil(2) is fugu's filesystem confinement), no daemonization, no
-mesh. Workers connect to the coordinator alone, which keeps the
-authority map auditable in one header (`proto.h`, I10).
+mesh. Workers' imsg channels connect to the coordinator alone, which keeps the
+authority map auditable in one header (`proto.h`, I10). The typed Artifact
+pipe relay is also enumerated there and is not a general Worker channel.
## Considered options
blob - da7a636f413daa37ad766b186a121ad2c9e29a87
blob + 1efd39fabc68a246908c0437b89e898c27d81a67
--- docs/adr/0006-typed-imsg-exchanges.md
+++ docs/adr/0006-typed-imsg-exchanges.md
The protocol test surface injects failure at every chunk and terminal position
and traces wrong sizes, out-of-order data, duplicate or missing terminals,
wrong correlation, overruns, and data after termination. This decision does
-not change ADR-0001's process topology, ADR-0002's libevent dispatch, or
-ADR-0003's bespoke hub-and-spoke privsep Module.
+not otherwise change ADR-0001's process topology, ADR-0002's libevent
+dispatch, or ADR-0003's bespoke hub-and-spoke privsep Module.
+
+The Artifact stream is the one narrow descriptor-bearing exception to normal
+`imsgev` traffic. `imsgev` owns an exact typed FD send/receive operation that
+requires the expected message type, correlation, payload size, descriptor
+presence, and close-on-exec state. `fugu-artifact` creates a pipe and sends
+only its producer endpoint to the Coordinator; the Coordinator validates and
+relays it only to the persistent Lead `fugu-tool`. OpenBSD's bidirectional
+pipe extension makes the exact endpoint access mode `O_RDWR`, not
+`O_WRONLY`; one-way use is a Role/protocol rule, while the descriptor check
+requires FIFO, blocking, and `CLOEXEC` state. No storage descriptor or path
+is transferred, ordinary messages still reject descriptors, and every
+unexpected, missing, reordered, or injected descriptor fails closed.
blob - bc52461e2f27a308cb13af3f73f3e345053213db
blob + d8f057b44600da6ccbae0c312dca346b6b931a17
--- docs/design/m10-release.md
+++ docs/design/m10-release.md
The source build installs:
- `${PREFIX}/bin/fugu`, mode 2555 and group `_fugu`;
-- `fugu-tty`, `fugu-api`, `fugu-web`, `fugu-tool`, and `fugu-editor` below
+- `fugu-tty`, `fugu-api`, `fugu-web`, `fugu-tool`, `fugu-artifact`, and
+ `fugu-editor` below
`${PREFIX}/libexec/fugu/`;
- `fugu.1` and `fugu.conf.5` in the normal manual hierarchy.
blob - e8324d9fadbfe9d0d87233da03609c5638756493
blob + 0e0c253057e1abd97e8b6ddbebf68b4e014e668a
--- docs/design/m3-privsep.md
+++ docs/design/m3-privsep.md
proofs. Real functionality (turns, tools, streaming) arrives in M4+;
M3 proves the *walls*.
+> **Post-M3 topology amendment:** ADR-0001 now adds the persistent
+> `fugu-artifact` Role for Owner-private retained Tool output. The original
+> milestone counts below are historical M3 scope, not the complete current
+> topology. Artifact control stays hub-and-spoke; one Coordinator-brokered,
+> one-way `fugu-tool` to `fugu-artifact` pipe protocol is the sole direct
+> Worker data plane. OpenBSD reports both pipe endpoints as `O_RDWR`; Role
+> ownership fixes the direction. ADR-0001, ADR-0003, and ADR-0006 define its
+> current authority and typed-descriptor rules.
+
## 1. Framework choice: a lean bespoke privsep module
vmd's `proc.c`/`proc.h` is a 730-line daemon framework: a full
socket, chroot, `daemon(3)`, config reload, `getpeereid` ACLs. fugu
is none of those things — it is a foreground, single-session,
per-invocation program with a **hub-and-spoke** authority map
-(ADR-0001): the coordinator talks to each worker; workers never talk
-to each other.
+(ADR-0001): the coordinator exchanges control with each worker; workers have
+no direct imsg channel.
**Decision:** write a small bespoke `priv` module that copies vmd's
*discipline* — the exact spawn ordering, fd pinning, `closefrom`,
├─ fork+exec libexec/fugu/fugu-api persistent, model-key custodian
├─ fork+exec libexec/fugu/fugu-web persistent, search custodian + blast room
├─ fork+exec libexec/fugu/fugu-tool persistent, tool executor
+ ├─ fork+exec libexec/fugu/fugu-artifact persistent, private output store
├─ fork+exec libexec/fugu/fugu-tool -r on demand, read-only, per subagent (M9)
└─ fork+exec libexec/fugu/fugu-editor on demand, compose room (M4/M7)
```
Every receiver validates type and length before use (I10); a
malformed message is a clean shutdown, never memory corruption. The
`imsgbuf_*` post-rewrite API from M-refs §4. Workers accept messages
-**only** from the coordinator fd; there is no worker-to-worker path
-to authorize.
+**only** from the coordinator fd. The later Artifact amendment adds no direct
+imsg path: it adds only the typed, correlated, one-way raw-byte pipe described
+above.
## 5. Teardown (I12)
blob - d246d795a3bbf6ae26e363738c23e91767f82a41
blob + ca9e3a2eff0c37a2e68c89d29cceba02314cb2a0
--- docs/design/m5-tools.md
+++ docs/design/m5-tools.md
rename, preserving the existing mode; offered only under
`allow_write`. `multi_edit` applies 1–64 pairs to an in-memory copy,
all-or-nothing, naming the failing 1-based index.
-- **shell** — `/bin/ksh -c` in the project dir, combined output
- capped at 256 KiB, 30 s wall clock then process-group kill.
+- **shell** — `/bin/ksh -c` in the project dir, combined output captured by
+ the private Artifact role under the 16 MiB stream and bounded generated-
+ result policy in behavior.md §5; 30 s wall clock then process-group kill.
- **grep** / **find** / **ls** — `grep -RnI`, `find`, `ls -la`
- semantics, 256 KiB caps.
+ semantics under the same Artifact capture policy.
- **shell_bg** / **shell_output** / **shell_kill** — a job table;
- each job in its own process group, exempt from the 30 s clock, a
- 256 KiB ring and bounded drain work per pump; `max_bg_jobs` cap
- returns a clean error rather than queueing. A direct-child
+ each job in its own process group, exempt from the 30 s clock. Each
+ `shell_output` freezes one immutable Artifact segment and gives a running job
+ a fresh segment; large results expose only a Session-local opaque id that
+ `output_read` can page. The `max_bg_jobs` cap returns a clean error rather
+ than queueing. A direct-child
supervisor anchors each command group until the worker reaps it,
preserves the command's exact wait status, and watches a worker-
liveness pipe, so ordinary group descendants die even if
Turn BEGIN/COMMIT/ABORT controls and are killed before an abandoned
Turn is reported (I12/I14, best-effort — `setsid(2)` escapes).
+The initial M5 baseline used direct 256 KiB result caps and a 256 KiB
+background ring. The current Artifact broker supersedes those storage details;
+the confinement, process-group, and Turn-transaction mechanics above remain
+the M5 foundation. The pinned current interface and limits live in
+behavior.md §5.
+
### File freshness
A per-process table records dev/ino/size/mtime as `read` returns each
blob - 77a25f0e2f957ef4bd6cbd5c4abc8c83b170b3e5
blob + 0bd4f50cce36898fe62ae23a36fb4db7975a95f6
--- docs/design/m9-subagents.md
+++ docs/design/m9-subagents.md
The OpenBSD `regress/agent` suite passes 155 checks. This slice adds the
model-facing `agent` tool, its Owner-bounded route palette, fresh read-only
workers, bounded execution, and the lead-only journal boundary required by
-behavior.md §5.12 and invariant I15. The combined normal and hardened release
+behavior.md §5.13 and invariant I15. The combined normal and hardened release
gates also pass as recorded by `docs/design/m10-release.md`.
## Surface and routing
blob - e8af57475d109b5c013829db53b40a12098fc695
blob + 95e57a866d9ee863fd031631e45cc6b7811a5124
--- docs/release-live-checks.md
+++ docs/release-live-checks.md
leaves it installed. It verifies:
1. package creation of `_fugu` with GID 992;
-2. the root/group/mode contract for the set-group-ID front binary, all five
+2. the root/group/mode contract for the set-group-ID front binary, all six
workers, both manuals, and `/etc/fugu.conf.sample`;
3. installation of every key and the supplied configuration as `root:_fugu`
mode 0640, while the ordinary user cannot read them directly;
blob - d1b228663ae914c56bda016364fe0548959da2a0
blob + 3499d090bca621c22c42b1227432d2af1e749ce4
--- handoff/README.md
+++ handoff/README.md
decisions by the author: strict behavior parity, known-flaw fixes
promoted to requirements (invariants.md I13/I14), and a set of
deliberate additions beyond parity — the kernel-confined subagent
-tool with per-call model routing (behavior.md §5.12, invariants.md
+tool with per-call model routing (behavior.md §5.13, invariants.md
I15), and a harness feature set chosen after a gap analysis against
contemporary agent harnesses: project context files with `#` capture,
bracketed paste, external-editor compose, a message queue with
blob - f78265824442903f56ca11c1e2cbcc7e11974c1a
blob + 156955f32d3f03a8c55b72479053527f54b833fa
--- handoff/behavior.md
+++ handoff/behavior.md
directory; absolute paths within it are fine. Every tool result is
bounded; every boundary value obeys invariants.md I13.
+The raw stdout/stderr of `shell`, `grep`, `find`, `ls`, `shell_bg`, and
+`shell_output` is brokered through the private Artifact role, not a Tool-owned
+ring buffer. Each captured stream retains at most 16 MiB. A complete generated
+result is inline only when its serialized document fits 256 KiB; otherwise it
+contains bounded head/tail previews and an opaque, Session-local output id.
+Private paths never cross the role boundary. `output_read` pages that id by raw
+byte offset (64 KiB default, 128 KiB maximum), as UTF-8 text or exact base64.
+
Always offered:
1. **read** `{path, offset?, limit?}` (required: path) — return file
request. Line scanning is length-aware end to end (I13).
Description steers: read before editing, use `ls` for directories,
read only the window needed.
-2. **write** `{path, content}` (required: both) — create or fully
+2. **output_read** `{id, offset?, limit?, encoding?}` (required: id) —
+ Lead-only; read one bounded raw-byte page from a Session-local opaque output
+ id.
+ `offset` defaults to 0, `limit` defaults to 64 KiB and is capped at
+ 128 KiB, and `encoding` is `text` (default) or `base64`. Text returns only
+ a valid UTF-8 prefix and directs the model to retry the same cursor as
+ base64 when necessary. Results report returned/total bytes, next offset,
+ EOF, and whether the retained object is pending in the current Turn or
+ committed and available. Unknown, expired, malformed, and out-of-range ids
+ are clean generated errors; ids are never paths.
+3. **write** `{path, content}` (required: both) — create or fully
overwrite; atomic (temp + fsync + rename), preserving an existing
file's mode; description steers: prefer `edit` for changes, don't
create unasked documentation. Offered only when `allow_write` is
on.
-3. **edit** `{path, old_string, new_string}` (required: all) — replace
+4. **edit** `{path, old_string, new_string}` (required: all) — replace
exactly one occurrence; fail if `old_string` is absent or
ambiguous; description steers: byte-exact match including
indentation, keep the match minimal but unique. Offered only when
`allow_write` is on.
-4. **shell** `{command}` (required: command) — run via `/bin/ksh -c`
- in the project directory; combined stdout+stderr, capped at
- 256 KiB with a truncation marker; 30-second wall clock, then kill;
+5. **shell** `{command}` (required: command) — run via `/bin/ksh -c`
+ in the project directory; combined stdout+stderr uses the 16 MiB Artifact
+ capture and generated-result policy above; 30-second wall clock, then kill;
description must state plainly that there is no network access and
name the web tools as the alternative, and warn about destructive
commands.
-5. **grep** `{pattern, path?}` — regex search over file contents
+6. **grep** `{pattern, path?}` — regex search over file contents
(recursive, line-numbered, binary-skipping, `grep -RnI`
- semantics), output capped at 256 KiB.
-6. **find** `{path?, name?}` — list files, optional glob filter,
- capped at 256 KiB.
-7. **ls** `{path?}` — long listing (`ls -la` semantics), capped at
- 256 KiB.
+ semantics), under the same Artifact capture policy.
+7. **find** `{path?, name?}` — list files, optional glob filter, under the
+ same Artifact capture policy.
+8. **ls** `{path?}` — long listing (`ls -la` semantics), under the same
+ Artifact capture policy.
Offered while `web_search` is on (default):
-8. **web_search** `{query}` (required) — brokered web search returning
+9. **web_search** `{query}` (required) — brokered web search returning
titles, URLs, snippets (reference: Kagi, 10 results, per-field
caps). The search provider sits behind a seam: `search_provider`
names the backend, "kagi" is the one that exists.
-9. **web_fetch** `{url}` (required) — HTTPS-only fetch returning
+10. **web_fetch** `{url}` (required) — HTTPS-only fetch returning
readable text (HTML reduced to text); raw body cap 2 MiB, returned
text cap 256 KiB with truncation marker; 30-second deadline; does
not follow redirects — a redirect status is reported to the model
Offered only when `http_allow` is configured:
-10. **http_request** `{url, method?, headers?, body?}` (required:
+11. **http_request** `{url, method?, headers?, body?}` (required:
url) — arbitrary-method HTTPS request with model-supplied headers
and body to hosts passing the allow/block patterns (§6); response
is the status plus body, non-2xx is a result rather than an
Offered when any model-invocable skill exists:
-11. **skill** `{name}` (required) — §4.
+12. **skill** `{name}` (required) — §4.
New in this revision, offered while `max_subagents` > 0 (default):
-12. **agent** `{prompt, label?, model?}` (required: prompt) — spawn a
+13. **agent** `{prompt, label?, model?}` (required: prompt) — spawn a
subagent: a fresh conversation with the same system conventions as
the lead, whose task is the prompt and whose final assistant text
is returned as the tool result. Constraints (also invariants.md
Offered when `allow_write` is on (with write and edit):
-13. **multi_edit** `{path, edits}` (required: both) — `edits` is an
+14. **multi_edit** `{path, edits}` (required: both) — `edits` is an
array of 1–64 `{old_string, new_string}` objects applied in order
to an in-memory copy of the file, each `old_string` required to
match exactly once in the intermediate content produced by the
Offered while `max_bg_jobs` > 0 (default), never to subagents:
-14. **shell_bg** `{command}` (required) — start a command under
+15. **shell_bg** `{command}` (required) — start a command under
exactly `shell`'s confinement and working directory, in its own
process group, exempt from the 30-second clock; returns a job id.
At the `max_bg_jobs` cap a further call returns a clean error
(builds, test suites); poll with `shell_output` instead of
re-running; kill jobs no longer needed; the same no-network
warning as `shell` applies.
-15. **shell_output** `{id}` (required) — return and consume the job's
- accumulated stdout+stderr (bounded ring buffer, 256 KiB per poll,
- overflow marked) plus running/exited status and the exit code
- once exited.
-16. **shell_kill** `{id}` (required) — terminate the job's process
+16. **shell_output** `{id}` (required) — atomically freeze and return the
+ job's accumulated Artifact segment plus running/exited status and the exit
+ code once exited. A still-running job adopts a fresh 16 MiB segment; the
+ frozen segment follows the same inline/opaque-id policy above. Repeated
+ polls therefore advance immutable segments rather than overwriting a ring.
+17. **shell_kill** `{id}` (required) — terminate the job's process
group.
Background jobs are session-scoped: a job spawned by a turn that
| `web_search` yes\|no | offer web_search + web_fetch [yes] |
| `search_provider` *string* | search backend name [kagi] |
| `max_subagents` *number* | concurrent subagent cap; 0 withholds the agent tool [4] |
-| `subagent_model` *string* | model id subagents use by default [unset: the lead's active model]; joins the agent tool's per-call palette (§5.12) |
+| `subagent_model` *string* | model id subagents use by default [unset: the lead's active model]; joins the agent tool's per-call palette (§5.13) |
| `subagent_provider` *string* | name of a configured provider block (or the implicit default) whose route and credential `subagent_model` is resolved against [the lead's active provider] |
| `subagent_effort` *string* | exact child Generation policy [unset: inherit `effort`; both unset: Provider default] |
| `max_bg_jobs` *number* | concurrent background shell jobs; 0 withholds shell_bg/shell_output/shell_kill [4] |
blob - 282a7a9294c4609250a0ac8875ecd2a563b5c6e2
blob + 2d2bf3039539bec1b6b06011388c3eae84bc39f2
--- handoff/invariants.md
+++ handoff/invariants.md
peer must produce a clean error or shutdown, not memory corruption.
IPC uses `imsg(3)`. Whatever the topology, the design must make each
role's *authority* — who can ask whom for what — explicit and
- auditable in one place.
+ auditable in one place. Fugu's control topology is hub-and-spoke. Its sole
+ direct Worker data plane is a bounded one-way protocol over a pipe from the
+ persistent Lead Tool Role to the Artifact Role for one correlated output
+ stream; the Coordinator validates and relays the typed producer endpoint,
+ ordinary imsg traffic rejects descriptors, and no storage descriptor or
+ path crosses the boundary. OpenBSD exposes both pipe endpoints as `O_RDWR`,
+ so Role ownership — not a false write-only claim — fixes the direction. The
+ Subagent Tool Role never accepts this stream.
- **I11 — least linkage.** Each executable links only the libraries its role
needs: libtls only where TLS is spoken and curses only where the terminal is
drawn. No ports library or language-runtime dependency may appear anywhere
escape the group — §4 records this non-guarantee).
Session journals are the Owner's own files, mode 0600, under
`~/.fugu/sessions/`; the role that writes them is confined to that
- directory at the time it writes.
+ directory at the time it writes. Retained raw Tool output is stored under
+ the separate Owner-only managed root `~/.fugu/artifacts/`; only the Artifact
+ Role can open it, with mode-0700 Session directories and mode-0600 files.
### Construction requirements
blob - a02345bf80cf8a1ddf2f55a57abc50758ab09fc8
blob + f61986f71509bbd3138c9bcdb0c69c885e25cd03
--- handoff/verification.md
+++ handoff/verification.md
file byte-identical and names the index.
- **Background jobs:** `max_bg_jobs` honored; jobs die on turn
failure, cancellation, session end, and drawer switch; unknown and
- prior-session ids error cleanly; ring-buffer overflow marked.
+ prior-session ids error cleanly. Repeated `shell_output` polls freeze exact
+ immutable Artifact segments, preserve unavailable reservations, and retire
+ available/unavailable spools through exact DROP/ACK, RELEASE, and job-wide
+ CANCEL/ACK races without reusing historical ownership.
- **Retry policy:** transport retries never occur after the first content delta;
`Retry-After` clamped to [1, 60] and parsed as untrusted (I13);
cancellation during backoff leaves the journal clean. Separately, exact
- **Read paging:** offset/limit windows respect the byte cap against
a newline-free 1 MiB line; totals and next-offset reported; zero
and negative values rejected.
+- **Artifact output paging and lifetime:** `artifact_result` computes the
+ complete serialized 256 KiB boundary and its +1 transition for printable,
+ escape-heavy, NUL, and invalid-UTF-8/base64 shapes. `artifact_page` pins raw
+ byte cursors, UTF-8 prefix behavior, base64 fidelity, and the 128 KiB page
+ cap. `artifact_store` covers pending/available/expired reads, 16 MiB stream,
+ 128 MiB Session and 1 GiB global quotas, reconciliation, eviction, and
+ transaction rollback. `turn` drives a real opaque id through Provider
+ `output_read`, Coordinator parsing/defaults, the Artifact role, an exact
+ NUL/invalid-byte base64 page, and the next Generation; a delayed response is
+ cancelled to prove its exact DATA/READ_RESULT drain precedes rollback.
- **Edit-diff display:** the display cap and elision marker
exercised; binary and over-cap content degrades to a summary;
hostile diff bytes are length-checked at the role boundary (I13);
blob - ea73a765a80c0fbb62e6ba01f1aa49dce9d7442a
blob + 02d3a12f946b134c4795cbd4a232f25c2eca37f0
--- port/productivity/fugu/pkg/PLIST
+++ port/productivity/fugu/pkg/PLIST
@group
libexec/fugu/
@bin libexec/fugu/fugu-api
+@bin libexec/fugu/fugu-artifact
@bin libexec/fugu/fugu-editor
@bin libexec/fugu/fugu-tool
@bin libexec/fugu/fugu-tty
blob - bb8099191815ac9e733c651c79e7f16af2d0c280
blob + 377dee31422ad5ea47f3498150895217538f7689
--- regress/README
+++ regress/README
(a hostile second terminal cannot
flip a committed turn; a tool
subprocess cannot reach the
- coordinator control fd); sandbox
+ coordinator control fd; normal exact
+ Artifact rotation/retirement and delayed
+ read frames cross the live role mesh);
+ tools (hostile Artifact rotation/drop/cancel
+ mechanics); artifact_store, artifact_page,
+ artifact_lock; sandbox
(fdguard: closefrom closes the
inherited control fd)
I11 least linkage deploy (installed ELF NEEDED sets and
sse (NUL transparency),
anthropic, openai, http, web, output
(NUL-bearing framing, headers, bodies,
- and JSON emission); boundary-wide
+ and JSON emission); artifact_result
+ (whole-document printable/escape/NUL/base64
+ boundaries), artifact_page (raw-byte text and
+ base64 cursors), turn (binary output_read);
+ boundary-wide
property grows with each
subsystem
I14 resume fidelity journal (transactional replay:
tools (executors, caps, freshness,
NUL rejection, bounded background
draining, supervisor/liveness cleanup);
+ artifact_result, artifact_page, artifact_store,
+ artifact_lock (brokered output retention and
+ paging); turn (opaque-id output_read loop and
+ cancelled exact-response drain);
web (search/fetch/request policy,
caps, credential scoping); turn
(full file loop); agent (ephemeral-
src/fugu/parse.y, conf.c conf
src/fugu/agentcfg.c agentcfg; agent (live schema/route)
src/fugu/priv.c, coord.c sandbox (spawn/handshake/teardown),
- turn, agent (lead and subagent turn loops)
+ turn (Lead loop and Coordinator Artifact
+ paging seam), agent (subagent turn loop)
+src/fugu/artifact_result.c,
+ artifact_capture.c artifact_result; tools; turn
+src/fugu/artifact_page.c artifact_page; turn (real output_read)
+src/fugu/artifact_lock.c,
+ artifact_scope.c,
+ artifact_pipe.c artifact_lock; sandbox; turn
+src/fugu-artifact (main.c, store.c) artifact_store; sandbox; turn
src/fugu/journal.c journal, turn
src/fugu/turn_txn.c turn_txn (pure ordering plus real-Journal
integration); turn (cache/SIGINT path)
blob - 48f9dca49e20ad3dff78f57dbfce466d63901fea
blob + 610e69cacd582d67d25bb12662536703ab9cfc8d
--- regress/agent/run.sh
+++ regress/agent/run.sh
lx=$dir/libexec/fugu
mkdir -p "$lx"
-for w in fugu-tty fugu-api fugu-web fugu-tool fugu-editor; do
+for w in fugu-tty fugu-api fugu-web fugu-tool fugu-artifact fugu-editor; do
b=$root/src/$w/obj/$w
[ -x "$b" ] || b=$root/src/$w/$w
[ -x "$b" ] || { echo "FAIL - $w not built"; exit 1; }
ok "agent loop: child receives the complete read-only tool surface" \
"$readonly_ok"
forbidden=0
-for name in edit write shell_bg shell_output agent skill http_request; do
+for name in edit write shell_bg shell_output output_read agent skill http_request; do
if grep -q "\"name\":\"$name\"" "$dir/child-initial"; then
forbidden=1
fi
blob - cd2e39e3a1622d80751e8bcebdf12cdda13a70ef
blob + d2c0f0ef3da80f624fbc914e2aec87cec80886c5
--- regress/artifact_lock/artifact_lock_test.c
+++ regress/artifact_lock/artifact_lock_test.c
*/
#include <sys/types.h>
+#include <sys/file.h>
#include <sys/stat.h>
#include <err.h>
+#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#include <stdio.h>
{
char base[] = "/tmp/fugu-artifact-lock.XXXXXXXXXX";
char session[PATH_MAX], lock[PATH_MAX], wrong[PATH_MAX];
- int fd, ro, other;
+ int contender, duplicate, fd, ro, third, wrongfd;
if (mkdtemp(base) == NULL ||
(size_t)snprintf(session, sizeof(session), "%s/session", base) >=
(fd = open(lock, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600)) == -1)
err(1, "create lock fixture");
CHECK(artifact_lifetime_lock_valid(fd, base, "session"));
+ contender = open(lock, O_RDWR | O_CLOEXEC);
+ CHECK(contender != -1 &&
+ !artifact_lifetime_lock_valid(contender, base, "session"));
+ duplicate = fcntl(fd, F_DUPFD_CLOEXEC, 0);
+ CHECK(duplicate != -1 &&
+ artifact_lifetime_lock_valid(duplicate, base, "session"));
ro = open(lock, O_RDONLY | O_CLOEXEC);
CHECK(ro != -1 && !artifact_lifetime_lock_valid(ro, base, "session"));
if ((size_t)snprintf(wrong, sizeof(wrong), "%s/wrong", session) >=
sizeof(wrong))
errx(1, "wrong-file fixture path is too long");
- other = open(wrong, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
- CHECK(other != -1 &&
- !artifact_lifetime_lock_valid(other, base, "session"));
- if (other != -1)
- close(other);
+ wrongfd = open(wrong, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+ CHECK(wrongfd != -1 &&
+ !artifact_lifetime_lock_valid(wrongfd, base, "session"));
+ if (wrongfd != -1)
+ close(wrongfd);
CHECK(fchmod(fd, 0644) == 0 &&
!artifact_lifetime_lock_valid(fd, base, "session"));
CHECK(fchmod(fd, 0600) == 0 &&
artifact_lifetime_lock_valid(fd, base, "session"));
+ if (duplicate != -1)
+ close(duplicate);
close(fd);
+ CHECK(contender != -1 &&
+ artifact_lifetime_lock_valid(contender, base, "session"));
+ third = open(lock, O_RDWR | O_CLOEXEC);
+ errno = 0;
+ CHECK(third != -1 && flock(third, LOCK_EX | LOCK_NB) == -1 &&
+ (errno == EWOULDBLOCK || errno == EAGAIN));
+ if (third != -1)
+ close(third);
+ if (contender != -1)
+ close(contender);
(void)unlink(wrong);
(void)unlink(lock);
(void)rmdir(session);
blob - 65b8df500be7aa4ca4d5e29cc597f05f5c0d63b8
blob + ae44b759a6d7e7e67be0de4889def77679866826
--- regress/artifact_page/artifact_page_test.c
+++ regress/artifact_page/artifact_page_test.c
#include "artifact_page.h"
#include "artifact_result.h"
#include "buf.h"
+#include "json.h"
#include "proto.h"
#include "regress.h"
static const char id[] = "0123456789abcdef0123456789abcdef";
+static int
+keys_exact(const struct buf *out, const char *const *want, size_t nwant)
+{
+ unsigned int seen = 0;
+ struct json j;
+ char *key;
+ size_t keylen, i;
+ int at, root, value, n, ok = 0;
+
+ if (nwant > sizeof(seen) * 8 ||
+ json_parse(&j, out->data, out->len, 0) == -1)
+ return (0);
+ root = json_root(&j);
+ n = json_obj_len(&j, root);
+ if (n < 0 || (size_t)n != nwant)
+ goto done;
+ for (at = 0; at < n; at++) {
+ if (json_obj_item(&j, root, at, &key, &keylen, &value) == -1)
+ goto done;
+ for (i = 0; i < nwant; i++)
+ if (strlen(want[i]) == keylen &&
+ memcmp(key, want[i], keylen) == 0)
+ break;
+ free(key);
+ if (i == nwant || (seen & (1U << i)) != 0)
+ goto done;
+ seen |= 1U << i;
+ }
+ ok = 1;
+done:
+ json_done(&j);
+ return (ok);
+}
+
int
main(void)
{
+ static const char *const page_keys[] = {
+ "id", "offset", "returned_bytes", "total_bytes", "next_offset",
+ "eof", "encoding", "state", "data", "is_error"
+ };
+ static const char *const retry_keys[] = {
+ "id", "offset", "encoding", "is_error", "error"
+ };
struct buf out;
struct artifact_read_req req;
struct artifact_read_result result;
u_char invalid[] = { 0x80, 'x' };
+ u_char invalid_after[] = { 'a', 0x80 };
u_char split[] = { 'a', 0xe2, 0x82 };
u_char *large;
int retry;
out.len <= ARTIFACT_RESULT_MAX &&
strstr(buf_cstr(&out), "\"data\":\"abc\"") != NULL &&
strstr(buf_cstr(&out), "\"eof\":true") != NULL);
+ CHECK(keys_exact(&out, page_keys,
+ sizeof(page_keys) / sizeof(page_keys[0])));
CHECK(artifact_page_build(&out, id, 7, 9, 0, invalid,
sizeof(invalid), ARTIFACT_PAGE_TEXT, &retry) == 0 && retry &&
out.len <= ARTIFACT_RESULT_MAX &&
strstr(buf_cstr(&out), "retry") != NULL &&
strstr(buf_cstr(&out), "base64") != NULL);
+ CHECK(keys_exact(&out, retry_keys,
+ sizeof(retry_keys) / sizeof(retry_keys[0])));
+ CHECK(artifact_page_build(&out, id, 0, sizeof(invalid_after), 0,
+ invalid_after, sizeof(invalid_after), ARTIFACT_PAGE_TEXT,
+ &retry) == 0 && !retry &&
+ strstr(buf_cstr(&out), "\"data\":\"a\"") != NULL &&
+ strstr(buf_cstr(&out), "\"returned_bytes\":1") != NULL &&
+ strstr(buf_cstr(&out), "\"next_offset\":1") != NULL);
+ CHECK(artifact_page_build(&out, id, 1, sizeof(invalid_after), 0,
+ invalid_after + 1, 1, ARTIFACT_PAGE_TEXT, &retry) == 0 && retry);
/* A valid lead byte split by the requested ceiling advances only through
- * the complete prefix; an incomplete sequence at EOF is invalid. */
+ * the complete prefix. At EOF that prefix is still returned; retrying at
+ * the incomplete sequence itself produces the base64 instruction. */
CHECK(artifact_page_build(&out, id, 0, 4, 1, split, sizeof(split),
ARTIFACT_PAGE_TEXT, &retry) == 0 && !retry &&
strstr(buf_cstr(&out), "\"returned_bytes\":1") != NULL &&
strstr(buf_cstr(&out), "\"next_offset\":1") != NULL);
CHECK(artifact_page_build(&out, id, 0, sizeof(split), 0, split,
- sizeof(split), ARTIFACT_PAGE_TEXT, &retry) == 0 && retry);
+ sizeof(split), ARTIFACT_PAGE_TEXT, &retry) == 0 && !retry &&
+ strstr(buf_cstr(&out), "\"returned_bytes\":1") != NULL &&
+ strstr(buf_cstr(&out), "\"next_offset\":1") != NULL);
+ CHECK(artifact_page_build(&out, id, 1, sizeof(split), 0, split + 1, 2,
+ ARTIFACT_PAGE_TEXT, &retry) == 0 && retry);
large = malloc(FUGU_ARTIFACT_PAGE_MAX);
CHECK(large != NULL);
blob - 006eb8ff4b201d90e5b8013013e710f4d499ff8e
blob + e9e8de2e750b837e6cfee7a4a5018a4684f8cc2b
--- regress/artifact_result/Makefile
+++ regress/artifact_result/Makefile
PROG= artifact_result_test
-SRCS= artifact_result_test.c artifact_result.c artifact_capture.c json.c \
+SRCS= artifact_result_test.c artifact_result.c artifact_capture.c \
+ artifact_scope.c json.c \
utf8.c buf.c log.c xmalloc.c
CFLAGS+= -I${.CURDIR}/../../src/fugu
blob - d407d2e47701c030541091407f2567fb2a235b32
blob + 1090a5bf0d5dc1f8c9e76e79baba43a936f5db9a
--- regress/artifact_result/artifact_result_test.c
+++ regress/artifact_result/artifact_result_test.c
#include "artifact_result.h"
#include "artifact_capture.h"
+#include "artifact_scope.h"
#include "buf.h"
#include "json.h"
#include "log.h"
return (count);
}
+static int
+keys_exact(const struct json *j, int obj, const char *const *want,
+ size_t nwant)
+{
+ unsigned int seen = 0;
+ char *key;
+ size_t keylen, i;
+ int at, value, n = json_obj_len(j, obj);
+
+ if (n < 0 || (size_t)n != nwant || nwant > sizeof(seen) * 8)
+ return (0);
+ for (at = 0; at < n; at++) {
+ if (json_obj_item(j, obj, at, &key, &keylen, &value) == -1)
+ return (0);
+ for (i = 0; i < nwant; i++)
+ if (strlen(want[i]) == keylen &&
+ memcmp(key, want[i], keylen) == 0)
+ break;
+ free(key);
+ if (i == nwant || (seen & (1U << i)) != 0)
+ return (0);
+ seen |= 1U << i;
+ }
+ return (1);
+}
+
static char *
string_field(const struct json *j, int root, const char *name, size_t *len)
{
static void
test_inline_and_forgery(void)
{
+ static const char *const keys[] = {
+ "preview_head", "preview_tail", "observed_bytes",
+ "retained_bytes", "preview_abbreviated",
+ "artifact_cap_reached", "termination", "preview_encoding",
+ "is_error", "exit_status"
+ };
static const char raw[] = "ok\n\",\"id\":\"forged\",\"is_error\":true";
struct artifact_result in;
struct json j;
CHECK(out.len <= ARTIFACT_RESULT_MAX);
CHECK(json_parse(&j, out.data, out.len, 0) == 0);
root = json_root(&j);
+ CHECK(keys_exact(&j, root, keys, sizeof(keys) / sizeof(keys[0])));
CHECK(json_obj_get(&j, root, "id") == -1);
CHECK(field_count(&j, root, "is_error") == 1);
CHECK(field_count(&j, root, "observed_bytes") == 1);
test_encoded_budgets_and_id(void)
{
static const char id[] = "0123456789abcdef0123456789abcdef";
+ static const char *const keys[] = {
+ "id", "preview_head", "preview_tail", "observed_bytes",
+ "retained_bytes", "preview_abbreviated",
+ "artifact_cap_reached", "termination", "preview_encoding",
+ "is_error", "exit_status"
+ };
struct artifact_result in;
struct json j;
struct buf raw, out;
CHECK(out.len <= ARTIFACT_RESULT_MAX);
CHECK(json_parse(&j, out.data, out.len, 0) == 0);
root = json_root(&j);
+ CHECK(keys_exact(&j, root, keys, sizeof(keys) / sizeof(keys[0])));
head = string_field(&j, root, "preview_head", &hlen);
tail = string_field(&j, root, "preview_tail", &tlen);
encoding = string_field(&j, root, "preview_encoding", &elen);
}
static void
+test_exact_inline_boundaries(void)
+{
+ static const struct {
+ const char *name;
+ u_char byte;
+ const char *encoding;
+ size_t encoded_step;
+ } shapes[] = {
+ { "printable", 'x', "utf-8", 1 },
+ { "escape-heavy", '\1', "utf-8", 6 },
+ { "NUL", '\0', "utf-8", 6 },
+ { "invalid UTF-8/base64", 0xff, "base64", 4 }
+ };
+ struct artifact_result in;
+ struct json j;
+ struct buf out;
+ char *encoding;
+ u_char *raw;
+ size_t elen, hi, i, lo, mid;
+ int root;
+
+ raw = malloc(ARTIFACT_RESULT_MAX + 1);
+ CHECK(raw != NULL);
+ if (raw == NULL)
+ return;
+ buf_init(&out);
+ for (i = 0; i < sizeof(shapes) / sizeof(shapes[0]); i++) {
+ memset(raw, shapes[i].byte, ARTIFACT_RESULT_MAX + 1);
+ lo = 0;
+ hi = ARTIFACT_RESULT_MAX + 1;
+ while (lo + 1 < hi) {
+ mid = lo + (hi - lo) / 2;
+ if (artifact_result_inline_fits(raw, mid))
+ lo = mid;
+ else
+ hi = mid;
+ }
+ CHECK(lo != 0 && artifact_result_inline_fits(raw, lo));
+ CHECK(!artifact_result_inline_fits(raw, lo + 1));
+
+ memset(&in, 0, sizeof(in));
+ in.raw = raw;
+ in.raw_len = lo;
+ in.observed_bytes = lo;
+ in.retained_bytes = lo;
+ in.termination = ARTIFACT_TERM_EXIT;
+ CHECK(artifact_result_build(&out, &in) == 0 &&
+ out.len <= ARTIFACT_RESULT_MAX &&
+ ARTIFACT_RESULT_MAX - out.len < shapes[i].encoded_step);
+ CHECK(json_parse(&j, out.data, out.len, 0) == 0);
+ root = json_root(&j);
+ CHECK(json_get_bool(&j, json_obj_get(&j, root,
+ "preview_abbreviated")) == 0);
+ encoding = string_field(&j, root, "preview_encoding", &elen);
+ CHECK(encoding != NULL && elen == strlen(shapes[i].encoding) &&
+ memcmp(encoding, shapes[i].encoding, elen) == 0);
+ free(encoding);
+ json_done(&j);
+
+ in.raw_len = lo + 1;
+ in.observed_bytes = lo + 1;
+ in.retained_bytes = lo + 1;
+ in.id = good_id;
+ CHECK(artifact_result_build(&out, &in) == 0 &&
+ out.len <= ARTIFACT_RESULT_MAX);
+ CHECK(json_parse(&j, out.data, out.len, 0) == 0);
+ root = json_root(&j);
+ CHECK(json_get_bool(&j, json_obj_get(&j, root,
+ "preview_abbreviated")) == 1);
+ json_done(&j);
+ (void)shapes[i].name;
+ }
+ buf_free(&out);
+ free(raw);
+}
+
+static void
test_error_metadata_is_not_spoofable(void)
{
static const char detail[] = "spawn failed before capture";
free(raw);
}
+static void
+test_capture_scope_is_exact(void)
+{
+ static const char *included[] = {
+ "shell", "grep", "find", "ls", "shell_bg", "shell_output"
+ };
+ static const char *excluded[] = {
+ "read", "write", "edit", "multi_edit", "shell_kill",
+ "output_read", "web_search", "web_fetch", "http_request",
+ "skill", "agent"
+ };
+ size_t i;
+
+ for (i = 0; i < sizeof(included) / sizeof(included[0]); i++)
+ CHECK(artifact_capture_tool_name(included[i]));
+ for (i = 0; i < sizeof(excluded) / sizeof(excluded[0]); i++)
+ CHECK(!artifact_capture_tool_name(excluded[i]));
+}
+
int
main(void)
{
test_encoded_budgets_and_id();
test_binary_base64();
test_boundary_selection();
+ test_exact_inline_boundaries();
test_error_metadata_is_not_spoofable();
test_json_integer_range();
test_empty_streaming_capture_is_inline();
test_fixed_wire_id_validation();
test_escape_heavy_error_budget();
test_streamed_equivalence();
+ test_capture_scope_is_exact();
REGRESS_END();
}
blob - fb4bcb5e128ddfb756d0b0a464368112e5d40e92
blob + ecc4e21e840b8a0c66405e3b405f5bb7d4cd9d6c
--- regress/artifact_store/Makefile
+++ regress/artifact_store/Makefile
PROG= artifact_store_test
-SRCS= artifact_store_test.c store.c artifact_result.c json.c buf.c log.c \
- xmalloc.c
+SRCS= artifact_store_test.c store.c artifact_lock.c artifact_result.c json.c \
+ buf.c log.c xmalloc.c
CFLAGS+= -I${.CURDIR}/../../src/fugu \
-I${.CURDIR}/../../src/fugu-artifact
blob - ab2a9add66a46151f9a1347bf300b2b084f6ad34
blob + 00ff8f0b07ef8f1fe47232c0131c252992729b38
--- regress/artifact_store/artifact_store_test.c
+++ regress/artifact_store/artifact_store_test.c
#include <unistd.h>
#include "artifact_result.h"
+#include "artifact_lock.h"
#include "buf.h"
#include "proto.h"
#include "store.h"
write_fd, stream));
}
+static struct artifact_store *
+open_ready(const char *root, const char *session, int fresh)
+{
+ struct artifact_store *store;
+
+ store = artifact_store_open(root, session, fresh);
+ if (store != NULL && artifact_store_reconcile_finish(store) == -1) {
+ artifact_store_close(store);
+ store = NULL;
+ }
+ return (store);
+}
+
static int
begin_stream(struct artifact_store *store, uint32_t correlation,
struct artifact_pending **pending, int *write_fd,
return (rc);
}
+#ifdef ARTIFACT_STORE_TESTING
static void
+test_reconciliation_state(const char *root)
+{
+ struct artifact_store *store;
+ struct artifact_pending *pending;
+ struct artifact_stream stream;
+ struct artifact_record record;
+ struct artifact_read_req req;
+ struct artifact_read_result result;
+ struct buf out;
+ char id[FUGU_ARTIFACT_ID_BUFSZ];
+ int write_fd = -1;
+
+ store = artifact_store_open(root, "reconcile-state", 1);
+ CHECK(store != NULL);
+ memset(&record, 0, sizeof(record));
+ id_for(42, id);
+ strlcpy(record.id, id, sizeof(record.id));
+ CHECK(artifact_store_turn_begin(store) == -1);
+ buf_init(&out);
+ memset(&req, 0, sizeof(req));
+ strlcpy(req.id, id, sizeof(req.id));
+ CHECK(artifact_store_read(store, &req, &out, &result) == -1);
+ CHECK(artifact_store_reconcile_add(store, &record) == 0);
+ artifact_store_test_fail(ARTIFACT_STORE_FAULT_RECONCILE_FSYNC);
+ CHECK(artifact_store_reconcile_finish(store) == -1);
+ /* Failed END leaves the initial phase retryable. */
+ CHECK(artifact_store_reconcile_add(store, &record) == 0);
+ CHECK(artifact_store_reconcile_finish(store) == 0);
+ CHECK(artifact_store_reconcile_finish(store) == -1);
+ CHECK(artifact_store_reconcile_add(store, &record) == -1);
+ CHECK(artifact_store_read(store, &req, &out, &result) == 0 &&
+ result.status == ARTIFACT_READ_EXPIRED);
+
+ CHECK(artifact_store_turn_begin(store) == 0);
+ CHECK(begin_stream(store, 1, &pending, &write_fd, &stream) == 0);
+ /* Reconciliation can never be restarted around live staging. */
+ CHECK(artifact_store_reconcile_add(store, &record) == -1);
+ CHECK(artifact_store_reconcile_finish(store) == -1);
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ buf_free(&out);
+ artifact_store_close(store);
+}
+#endif
+
+static void
test_lock_and_permissions(const char *root)
{
struct artifact_store *store, *again;
struct stat st;
- char path[PATH_MAX];
+ char path[PATH_MAX], alias[PATH_MAX], target[PATH_MAX], outside[PATH_MAX];
pid_t pid;
int p[2], status;
char result;
store = artifact_store_open(root, "lock-session", 1);
CHECK(store != NULL);
+ CHECK(stat(root, &st) == 0 && st.st_uid == geteuid() &&
+ (st.st_mode & 0777) == 0700);
path_for(path, sizeof(path), root, "lock-session", ".session.lock");
- CHECK(stat(path, &st) == 0 && (st.st_mode & 0777) == 0600);
+ CHECK(stat(path, &st) == 0 && st.st_uid == geteuid() &&
+ (st.st_mode & 0777) == 0600 && st.st_nlink == 1);
if ((size_t)snprintf(path, sizeof(path), "%s/%s", root,
"lock-session") >= sizeof(path))
errx(1, "lock path");
- CHECK(stat(path, &st) == 0 && (st.st_mode & 0777) == 0700);
+ CHECK(stat(path, &st) == 0 && st.st_uid == geteuid() &&
+ (st.st_mode & 0777) == 0700);
+ CHECK(artifact_store_open(root, "", 0) == NULL && errno == EINVAL);
+ CHECK(artifact_store_open(root, "../foreign", 0) == NULL &&
+ errno == EINVAL);
+ CHECK(artifact_store_open(root, "bad/session", 0) == NULL &&
+ errno == EINVAL);
CHECK(artifact_store_open(root, "lock-session", 1) == NULL &&
errno == EEXIST);
if (pipe(p) == -1)
{
int lock_copy = artifact_store_lock_dup(store);
- CHECK(lock_copy != -1);
+ CHECK(lock_copy != -1 && artifact_lifetime_lock_valid(lock_copy,
+ root, "lock-session"));
artifact_store_close(store);
CHECK(artifact_store_open(root, "lock-session", 0) == NULL &&
errno == EWOULDBLOCK);
again = artifact_store_open(root, "lock-session", 0);
CHECK(again != NULL);
artifact_store_close(again);
+
+ /* Managed roots and Session names are exact directories, never aliases. */
+ CHECK((size_t)snprintf(alias, sizeof(alias), "%s-root-alias", root) <
+ sizeof(alias));
+ CHECK((size_t)snprintf(target, sizeof(target), "%s-root-target", root) <
+ sizeof(target));
+ CHECK(mkdir(target, 0700) == 0 && symlink(target, alias) == 0);
+ CHECK(artifact_store_open(alias, "session", 1) == NULL);
+
+ CHECK((size_t)snprintf(alias, sizeof(alias), "%s-session-alias", root) <
+ sizeof(alias));
+ CHECK((size_t)snprintf(target, sizeof(target), "%s-session-target", root) <
+ sizeof(target));
+ CHECK(mkdir(alias, 0700) == 0 && mkdir(target, 0700) == 0);
+ CHECK((size_t)snprintf(path, sizeof(path), "%s/session", alias) <
+ sizeof(path) && symlink(target, path) == 0);
+ CHECK(artifact_store_open(alias, "session", 0) == NULL);
+
+ /* A hard-linked lock could alias another Owner file; exact nlink rejects it. */
+ CHECK((size_t)snprintf(alias, sizeof(alias), "%s-lock-alias", root) <
+ sizeof(alias));
+ mkdir_session(alias, "session");
+ path_for(path, sizeof(path), alias, "session", ".session.lock");
+ CHECK((size_t)snprintf(outside, sizeof(outside), "%s/outside", alias) <
+ sizeof(outside));
+ {
+ int outside_fd = open(outside,
+ O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+
+ CHECK(outside_fd != -1);
+ if (outside_fd != -1)
+ close(outside_fd);
+ }
+ CHECK(unlink(path) == 0 && link(outside, path) == 0);
+ CHECK(artifact_store_open(alias, "session", 0) == NULL);
}
static void
char committed[FUGU_ARTIFACT_ID_BUFSZ], aborted[FUGU_ARTIFACT_ID_BUFSZ];
int write_fd, rc;
- store = artifact_store_open(root, "states", 1);
+ store = open_ready(root, "states", 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream(store, 1, &pending, &write_fd, &stream) == 0 &&
stream.status == ARTIFACT_STREAM_AVAILABLE);
}
static void
+test_session_read_isolation(const char *root)
+{
+ struct artifact_store *one, *two;
+ struct artifact_pending *pending;
+ struct artifact_stream stream;
+ struct artifact_terminal terminal;
+ struct artifact_record record;
+ struct artifact_read_req req;
+ struct artifact_read_result result;
+ struct buf out;
+ struct stat st;
+ char id[FUGU_ARTIFACT_ID_BUFSZ], source[PATH_MAX], alias[PATH_MAX];
+ int write_fd;
+
+ one = open_ready(root, "isolation-one", 1);
+ CHECK(one != NULL && artifact_store_turn_begin(one) == 0);
+ CHECK(begin_stream(one, 1, &pending, &write_fd, &stream) == 0 &&
+ write_and_drain(pending, write_fd, 3, &terminal) == 1 &&
+ artifact_store_turn_prepare(one) == 0 &&
+ artifact_store_turn_commit(one) == 0);
+ strlcpy(id, stream.id, sizeof(id));
+ path_for(source, sizeof(source), root, "isolation-one", id);
+ CHECK(stat(source, &st) == 0 && st.st_uid == geteuid() &&
+ (st.st_mode & 0777) == 0600 && st.st_nlink == 1);
+
+ two = artifact_store_open(root, "isolation-two", 1);
+ buf_init(&out);
+ memset(&req, 0, sizeof(req));
+ strlcpy(req.id, id, sizeof(req.id));
+ req.limit = 3;
+ /* Even corrupted replay knowledge cannot turn a same-Owner file alias
+ * into a cross-Session read: final symlinks and hard links are rejected. */
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = 3;
+ CHECK(artifact_store_reconcile_add(two, &record) == 0 &&
+ artifact_store_reconcile_finish(two) == 0);
+ path_for(alias, sizeof(alias), root, "isolation-two", id);
+ CHECK(symlink(source, alias) == 0 &&
+ artifact_store_read(two, &req, &out, &result) == 0 &&
+ result.status == ARTIFACT_READ_ERROR && out.len == 0 &&
+ unlink(alias) == 0);
+ CHECK(link(source, alias) == 0 &&
+ artifact_store_read(two, &req, &out, &result) == 0 &&
+ result.status == ARTIFACT_READ_ERROR && out.len == 0 &&
+ unlink(alias) == 0);
+ buf_free(&out);
+ artifact_store_close(two);
+ artifact_store_close(one);
+}
+
+static void
test_spool_ownership(const char *root)
{
struct artifact_store *store;
char detached[FUGU_ARTIFACT_ID_BUFSZ];
int write_fd;
- store = artifact_store_open(root, "spools", 1);
+ store = open_ready(root, "spools", 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream_kind(store, 1, ARTIFACT_BEGIN_SPOOL, &pending,
&write_fd, &stream) == 0);
struct artifact_terminal terminal;
int fdone, fdtwo, ignored_fd = -1;
- store = artifact_store_open(root, "correlations", 1);
+ store = open_ready(root, "correlations", 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream_kind(store, 10, ARTIFACT_BEGIN_SPOOL, &one,
&fdone, &sone) == 0);
char id[FUGU_ARTIFACT_ID_BUFSZ], path[PATH_MAX], byte = 0;
int fd, write_fd;
- store = artifact_store_open(root, "publication", 1);
+ store = open_ready(root, "publication", 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream(store, 1, &pending, &write_fd, &stream) == 0 &&
write_and_drain(pending, write_fd, 1, &terminal) == 1);
for (i = 0; i < sizeof(faults) / sizeof(faults[0]); i++) {
(void)snprintf(session, sizeof(session), "terminal-%zu", i);
- store = artifact_store_open(root, session, 1);
+ store = open_ready(root, session, 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream_kind(store, 1, ARTIFACT_BEGIN_SPOOL,
&pending, &write_fd, &stream) == 0);
for (i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) {
(void)snprintf(session, sizeof(session), "abort-%zu", i);
- store = artifact_store_open(root, session, 1);
+ store = open_ready(root, session, 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream(store, 1, &pending, &write_fd, &stream) == 0 &&
write_and_drain(pending, write_fd, 1, &terminal) == 1);
}
static void
+test_partial_publication_cleanup(const char *root)
+{
+ struct artifact_store *store;
+ struct artifact_pending *one, *two;
+ struct artifact_stream sone, stwo;
+ struct artifact_terminal terminal;
+ char pending[64], order[64], path[PATH_MAX], byte = 0;
+ int fd, fdone, fdtwo;
+
+ store = open_ready(root, "partial-publish-abort", 1);
+ CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
+ CHECK(begin_stream(store, 1, &one, &fdone, &sone) == 0 &&
+ write_and_drain(one, fdone, 3, &terminal) == 1);
+ artifact_store_test_fail(
+ ARTIFACT_STORE_FAULT_PREPARE_STAGING_UNLINK);
+ CHECK(artifact_store_turn_prepare(store) == -1);
+ (void)snprintf(pending, sizeof(pending), ".pending-%s", sone.id);
+ (void)snprintf(order, sizeof(order), ".order-%s", sone.id);
+ CHECK(exists(root, "partial-publish-abort", sone.id) &&
+ exists(root, "partial-publish-abort", pending));
+ CHECK(artifact_store_turn_abort(store) == 0);
+ CHECK(!exists(root, "partial-publish-abort", sone.id) &&
+ !exists(root, "partial-publish-abort", pending) &&
+ !exists(root, "partial-publish-abort", order));
+ artifact_store_close(store);
+
+ /* A later collision can follow an earlier successful publication in the
+ * same prepare. Shutdown rolls the whole partial prepare back without
+ * deleting the colliding pre-existing final. */
+ store = open_ready(root, "partial-publish-close", 1);
+ CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
+ CHECK(begin_stream(store, 1, &one, &fdone, &sone) == 0 &&
+ begin_stream(store, 2, &two, &fdtwo, &stwo) == 0);
+ CHECK(write_and_drain(one, fdone, 1, &terminal) == 1 &&
+ write_and_drain(two, fdtwo, 2, &terminal) == 1);
+ path_for(path, sizeof(path), root, "partial-publish-close", sone.id);
+ fd = open(path, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+ CHECK(fd != -1 && write(fd, "Z", 1) == 1);
+ if (fd != -1)
+ close(fd);
+ CHECK(artifact_store_turn_prepare(store) == -1 &&
+ exists(root, "partial-publish-close", stwo.id));
+ artifact_store_close(store);
+ (void)snprintf(pending, sizeof(pending), ".pending-%s", sone.id);
+ CHECK(!exists(root, "partial-publish-close", pending) &&
+ !exists(root, "partial-publish-close", stwo.id));
+ fd = open(path, O_RDONLY | O_CLOEXEC);
+ CHECK(fd != -1 && read(fd, &byte, 1) == 1 && byte == 'Z');
+ if (fd != -1)
+ close(fd);
+}
+
+static void
+test_metadata_validation_and_nonblocking_opens(const char *root)
+{
+ struct artifact_store *store;
+ struct artifact_pending *pending = NULL;
+ struct artifact_stream stream;
+ struct artifact_terminal terminal;
+ struct artifact_record record;
+ struct artifact_read_req req;
+ struct stat st;
+ char subroot[PATH_MAX], path[PATH_MAX], outside[PATH_MAX];
+ char id[FUGU_ARTIFACT_ID_BUFSZ], order[64];
+ uint64_t marker = UINT64_C(0x1122334455667788), observed = 0;
+ pid_t pid;
+ mode_t oldmask;
+ int fd, status, write_fd = -1, rc;
+
+ CHECK(mkdir(root, 0700) == 0);
+
+ /* Creation reaches exact private modes even when the inherited Owner
+ * umask masks every permission bit. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/creation-umask", root);
+ store = open_ready(subroot, "session", 1);
+ CHECK(store != NULL && artifact_store_turn_begin(store) == 0 &&
+ begin_stream(store, 1, &pending, &write_fd, &stream) == 0);
+ oldmask = umask(0777);
+ rc = write_and_drain(pending, write_fd, 1, &terminal);
+ (void)umask(oldmask);
+ CHECK(rc == 1 && terminal.status == ARTIFACT_STORE_OK);
+ (void)snprintf(path, sizeof(path), "%s/.sequence", subroot);
+ CHECK(stat(path, &st) == 0 && (st.st_mode & 0777) == 0600);
+ (void)snprintf(path, sizeof(path), "%s/session/.order-%s", subroot,
+ stream.id);
+ CHECK(stat(path, &st) == 0 && (st.st_mode & 0777) == 0600);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ /* A preplanted hard link must not let sequence allocation chmod or write
+ * an arbitrary Owner file. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/sequence", root);
+ store = open_ready(subroot, "session", 1);
+ CHECK(store != NULL && artifact_store_turn_begin(store) == 0 &&
+ begin_stream(store, 1, &pending, &write_fd, &stream) == 0);
+ (void)snprintf(outside, sizeof(outside), "%s/sequence-outside", root);
+ fd = open(outside, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0644);
+ CHECK(fd != -1 && fchmod(fd, 0644) == 0 &&
+ write(fd, &marker, sizeof(marker)) == (ssize_t)sizeof(marker));
+ if (fd != -1)
+ close(fd);
+ (void)snprintf(path, sizeof(path), "%s/.sequence", subroot);
+ CHECK(link(outside, path) == 0);
+ rc = write_and_drain(pending, write_fd, 1, &terminal);
+ CHECK(rc == -1 && terminal.status == ARTIFACT_STORE_ERROR);
+ fd = open(outside, O_RDONLY | O_CLOEXEC);
+ CHECK(fd != -1 && fstat(fd, &st) == 0 &&
+ (st.st_mode & 0777) == 0644 &&
+ pread(fd, &observed, sizeof(observed), 0) ==
+ (ssize_t)sizeof(observed) && observed == marker);
+ if (fd != -1)
+ close(fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ /* Existing order metadata is exact, including mode and link count. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/order-mode", root);
+ store = artifact_store_open(subroot, "session", 1);
+ id_for(100, id);
+ make_sparse(subroot, "session", id, 1, 1);
+ make_order(subroot, "session", id, 1);
+ (void)snprintf(order, sizeof(order), ".order-%s", id);
+ path_for(path, sizeof(path), subroot, "session", order);
+ CHECK(chmod(path, 0644) == 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = 1;
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0 &&
+ artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE);
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ (void)snprintf(subroot, sizeof(subroot), "%s/order-link", root);
+ store = artifact_store_open(subroot, "session", 1);
+ id_for(101, id);
+ make_sparse(subroot, "session", id, 1, 1);
+ (void)snprintf(outside, sizeof(outside), "%s/order-outside", root);
+ fd = open(outside, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+ CHECK(fd != -1 && write(fd, &marker, sizeof(marker)) ==
+ (ssize_t)sizeof(marker));
+ if (fd != -1)
+ close(fd);
+ (void)snprintf(order, sizeof(order), ".order-%s", id);
+ path_for(path, sizeof(path), subroot, "session", order);
+ CHECK(link(outside, path) == 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = 1;
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0 &&
+ artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE);
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ /* FIFOs at read-only path boundaries return an error promptly rather than
+ * waiting for an attacker-controlled peer. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/order-fifo", root);
+ store = artifact_store_open(subroot, "session", 1);
+ id_for(102, id);
+ make_sparse(subroot, "session", id, 1, 1);
+ (void)snprintf(order, sizeof(order), ".order-%s", id);
+ path_for(path, sizeof(path), subroot, "session", order);
+ CHECK(mkfifo(path, 0600) == 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = 1;
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0 &&
+ artifact_store_turn_begin(store) == 0);
+ pid = fork();
+ if (pid == 0) {
+ alarm(2);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ _exit(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE ?
+ 0 : 1);
+ }
+ CHECK(pid != -1 && waitpid(pid, &status, 0) == pid &&
+ WIFEXITED(status) && WEXITSTATUS(status) == 0);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ (void)snprintf(subroot, sizeof(subroot), "%s/final-fifo", root);
+ store = artifact_store_open(subroot, "session", 1);
+ id_for(103, id);
+ path_for(path, sizeof(path), subroot, "session", id);
+ CHECK(mkfifo(path, 0600) == 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0);
+ pid = fork();
+ if (pid == 0) {
+ struct artifact_read_result result;
+ struct buf out;
+
+ alarm(2);
+ buf_init(&out);
+ memset(&req, 0, sizeof(req));
+ strlcpy(req.id, id, sizeof(req.id));
+ _exit(artifact_store_read(store, &req, &out, &result) == 0 &&
+ result.status == ARTIFACT_READ_ERROR ? 0 : 1);
+ }
+ CHECK(pid != -1 && waitpid(pid, &status, 0) == pid &&
+ WIFEXITED(status) && WEXITSTATUS(status) == 0);
+ artifact_store_close(store);
+}
+
+static void
+fill_sparse_session(const char *root, const char *session, unsigned int base)
+{
+ char id[FUGU_ARTIFACT_ID_BUFSZ];
+ int i;
+
+ for (i = 0; i < 64; i++) {
+ id_for(base + (unsigned int)i, id);
+ make_sparse(root, session, id, ARTIFACT_RAW_MAX, 1 + i);
+ }
+}
+
+static void
+test_quota_metadata_and_inactive_cleanup(const char *root)
+{
+ struct artifact_store *store;
+ struct artifact_pending *pending = NULL;
+ struct artifact_stream stream;
+ struct artifact_record record;
+ struct stat st;
+ char subroot[PATH_MAX], path[PATH_MAX], outside[PATH_MAX];
+ char id[FUGU_ARTIFACT_ID_BUFSZ], first[FUGU_ARTIFACT_ID_BUFSZ];
+ char pending_name[64], order_name[64];
+ int fd, write_fd = -1, rc;
+
+ CHECK(mkdir(root, 0700) == 0);
+
+ /* Every final counted by quota has the same exact private-file shape as
+ * a final admitted by output_read. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/final-mode", root);
+ store = artifact_store_open(subroot, "current", 1);
+ id_for(200, id);
+ make_sparse(subroot, "current", id, 1, 1);
+ path_for(path, sizeof(path), subroot, "current", id);
+ CHECK(chmod(path, 0644) == 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = 1;
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0 &&
+ artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ exists(subroot, "current", id));
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ (void)snprintf(subroot, sizeof(subroot), "%s/final-link", root);
+ store = artifact_store_open(subroot, "current", 1);
+ id_for(201, id);
+ (void)snprintf(outside, sizeof(outside), "%s/final-outside", root);
+ fd = open(outside, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+ CHECK(fd != -1 && write(fd, "X", 1) == 1);
+ if (fd != -1)
+ close(fd);
+ path_for(path, sizeof(path), subroot, "current", id);
+ CHECK(link(outside, path) == 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = 1;
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0 &&
+ artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ exists(subroot, "current", id));
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ (void)snprintf(subroot, sizeof(subroot), "%s/final-oversize", root);
+ store = artifact_store_open(subroot, "current", 1);
+ id_for(202, id);
+ make_sparse(subroot, "current", id, ARTIFACT_RAW_MAX + 1, 1);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ record.retained_bytes = ARTIFACT_RAW_MAX;
+ CHECK(store != NULL && artifact_store_reconcile_add(store, &record) == 0 &&
+ artifact_store_reconcile_finish(store) == 0 &&
+ artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ exists(subroot, "current", id));
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ /* An inactive crash between final link and staging unlink is repaired as
+ * a distinct synchronized pass; its valid final and FIFO order survive. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/crash-link", root);
+ store = open_ready(subroot, "current", 1);
+ mkdir_session(subroot, "inactive");
+ id_for(300, id);
+ make_sparse(subroot, "inactive", id, 1, 1);
+ path_for(path, sizeof(path), subroot, "inactive", id);
+ (void)snprintf(pending_name, sizeof(pending_name), ".pending-%s", id);
+ path_for(outside, sizeof(outside), subroot, "inactive", pending_name);
+ CHECK(link(path, outside) == 0);
+ make_order(subroot, "inactive", id, 1);
+ (void)snprintf(order_name, sizeof(order_name), ".order-%s", id);
+ CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
+ artifact_store_test_fail(ARTIFACT_STORE_FAULT_INACTIVE_CLEANUP_FSYNC);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE);
+ if (rc == 0)
+ close(write_fd);
+ rc = begin_stream(store, 2, &pending, &write_fd, &stream);
+ CHECK(rc == 0 && !exists(subroot, "inactive", pending_name) &&
+ exists(subroot, "inactive", id) &&
+ exists(subroot, "inactive", order_name));
+ path_for(path, sizeof(path), subroot, "inactive", id);
+ CHECK(stat(path, &st) == 0 && st.st_nlink == 1);
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ /* Invalid Session directories, names, and lifetime locks are never made
+ * eviction candidates even under global pressure. */
+ (void)snprintf(subroot, sizeof(subroot), "%s/session-mode", root);
+ store = open_ready(subroot, "current", 1);
+ mkdir_session(subroot, "inactive");
+ fill_sparse_session(subroot, "inactive", 1000);
+ (void)snprintf(path, sizeof(path), "%s/inactive", subroot);
+ CHECK(chmod(path, 0755) == 0);
+ id_for(1000, first);
+ CHECK(artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ exists(subroot, "inactive", first));
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ (void)snprintf(subroot, sizeof(subroot), "%s/session-name", root);
+ store = open_ready(subroot, "current", 1);
+ mkdir_session(subroot, "invalid_name");
+ fill_sparse_session(subroot, "invalid_name", 2000);
+ id_for(2000, first);
+ CHECK(artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ exists(subroot, "invalid_name", first));
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+
+ (void)snprintf(subroot, sizeof(subroot), "%s/session-lock", root);
+ store = open_ready(subroot, "current", 1);
+ mkdir_session(subroot, "inactive");
+ fill_sparse_session(subroot, "inactive", 3000);
+ path_for(path, sizeof(path), subroot, "inactive", ".session.lock");
+ CHECK(unlink(path) == 0);
+ (void)snprintf(outside, sizeof(outside), "%s/lock-outside", root);
+ fd = open(outside, O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC, 0600);
+ CHECK(fd != -1);
+ if (fd != -1)
+ close(fd);
+ CHECK(link(outside, path) == 0);
+ id_for(3000, first);
+ CHECK(artifact_store_turn_begin(store) == 0);
+ rc = begin_stream(store, 1, &pending, &write_fd, &stream);
+ CHECK(rc == 1 && stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ exists(subroot, "inactive", first));
+ if (rc == 0)
+ close(write_fd);
+ CHECK(artifact_store_turn_abort(store) == 0);
+ artifact_store_close(store);
+}
+
+static void
test_discard_retry_does_not_close_reused_fd(const char *root)
{
struct artifact_store *store;
char path[PATH_MAX];
int sentinel, write_fd;
- store = artifact_store_open(root, "discard-retry", 1);
+ store = open_ready(root, "discard-retry", 1);
CHECK(store != NULL && artifact_store_turn_begin(store) == 0);
CHECK(begin_stream(store, 1, &pending, &write_fd, &stream) == 0 &&
write_and_drain(pending, write_fd, 1, &terminal) == 1);
close(result[0]);
close(stop[1]);
(void)snprintf(session, sizeof(session), "quota-%u", session_no);
- store = artifact_store_open(root, session, 1);
+ store = open_ready(root, session, 1);
if (store == NULL || artifact_store_turn_begin(store) == -1)
_exit(2);
(void)write(result[1], &ch, 1);
struct artifact_pending *pending;
struct artifact_stream stream;
char id[FUGU_ARTIFACT_ID_BUFSZ], first[FUGU_ARTIFACT_ID_BUFSZ];
- int write_fd, i;
+ int write_fd = -1, i;
- current = artifact_store_open(root, "current", 1);
+ current = open_ready(root, "current", 1);
CHECK(current != NULL);
mkdir_session(root, "active");
id_for(900, id);
char inactive_id[FUGU_ARTIFACT_ID_BUFSZ];
int i, write_fd = -1;
- current = artifact_store_open(root, "current", 1);
+ current = open_ready(root, "current", 1);
CHECK(current != NULL);
mkdir_session(root, "inactive");
id_for(2000, current_id);
{
char base[] = "/tmp/fugu-artifact-store.XXXXXXXXXX";
char root[PATH_MAX], quota[PATH_MAX], active[PATH_MAX], priority[PATH_MAX];
+ char metadata[PATH_MAX];
+ char quota_metadata[PATH_MAX];
if (mkdtemp(base) == NULL)
err(1, "mkdtemp");
(void)snprintf(quota, sizeof(quota), "%s/quota", base);
(void)snprintf(active, sizeof(active), "%s/active-root", base);
(void)snprintf(priority, sizeof(priority), "%s/priority", base);
+ (void)snprintf(metadata, sizeof(metadata), "%s/metadata", base);
+ (void)snprintf(quota_metadata, sizeof(quota_metadata),
+ "%s/quota-metadata", base);
test_lock_and_permissions(root);
+#ifdef ARTIFACT_STORE_TESTING
+ test_reconciliation_state(root);
+#endif
test_cap_and_transaction(root);
+ test_session_read_isolation(root);
test_spool_ownership(root);
test_correlation_keys(root);
test_publication_collision_and_prepared_close(root);
#ifdef ARTIFACT_STORE_TESTING
test_terminal_storage_errors(root);
test_abort_retry(root);
+ test_partial_publication_cleanup(root);
test_discard_retry_does_not_close_reused_fd(root);
+ test_metadata_validation_and_nonblocking_opens(metadata);
+ test_quota_metadata_and_inactive_cleanup(quota_metadata);
#endif
test_reconcile_and_fifo(root);
test_concurrent_global_quota(quota);
blob - a7bebf14d9e15b9ab01591791f276c797059063e
blob + 056675df21b268029c06cafe23d7c4eef2f1562f
--- regress/deploy/deploy_test.sh
+++ regress/deploy/deploy_test.sh
test "$(stat -f %Mp%Lp "$prefix/bin/fugu" 2>/dev/null)" = 2555
ok "layout: fugu is installed mode 2555" $?
-workers='fugu-api fugu-editor fugu-tool fugu-tty fugu-web'
+workers='fugu-api fugu-artifact fugu-editor fugu-tool fugu-tty fugu-web'
missing=0
badmode=0
for worker in $workers; do
test "$(stat -f %Lp "$prefix/libexec/fugu/$worker" 2>/dev/null)" = \
555 || badmode=1
done
-ok "layout: all five workers are under PREFIX/libexec/fugu" "$missing"
+ok "layout: all six workers are under PREFIX/libexec/fugu" "$missing"
ok "layout: workers are ordinary mode-0555 executables" "$badmode"
test -f "$prefix/man/man1/fugu.1" && \
test "$(needed "$prefix/libexec/fugu/fugu-web")" = "c event tls util"
ok "linkage: only network custodians add tls" $?
test "$(needed "$prefix/libexec/fugu/fugu-tool")" = "c event util" && \
+ test "$(needed "$prefix/libexec/fugu/fugu-artifact")" = "c event util" && \
test "$(needed "$prefix/libexec/fugu/fugu-editor")" = "c event util"
-ok "linkage: tool and editor map no tls or curses" $?
+ok "linkage: tool, artifact, and editor map no tls or curses" $?
for binary in "$prefix/bin/fugu" "$prefix/libexec/fugu"/*; do
needed "$binary"
blob - b6c459d77ce6bd555f8e8e3dca2f41fdab2e4bfd
blob + ec864a3123dbff7336905ecbdae77c205e6a3c04
--- regress/journal/journal_test.c
+++ regress/journal/journal_test.c
}
static void
+test_artifact_survives_epoch_resets(void)
+{
+ static const char artifact_id[] =
+ "0123456789abcdef0123456789abcdef";
+ struct system_epoch start, compact, clear;
+ struct artifact_record record;
+ struct journal_state state;
+ struct journal *j;
+ char id[SESSION_ID_MAX];
+
+ make_epoch(&start, SYSTEM_EPOCH_START, NULL, 0);
+ make_epoch(&compact, SYSTEM_EPOCH_COMPACT, "retained", 8);
+ make_epoch(&clear, SYSTEM_EPOCH_CLEAR, NULL, 0);
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, artifact_id, sizeof(record.id));
+ record.retained_bytes = 12345;
+
+ j = journal_create(dir, id, sizeof(id));
+ CHECK(j != NULL &&
+ journal_system_epoch(j, &start, NULL, NULL, 0) == 0);
+ journal_turn_begin(j, 1);
+ journal_artifact(j, &record);
+ append_text_message(j, ROLE_USER, "artifact owner", 14);
+ append_text_message(j, ROLE_ASSISTANT, "artifact result", 15);
+ journal_turn_end(j, 1, TURN_OK);
+ CHECK(journal_system_epoch(j, &compact, NULL, NULL, 0) == 0);
+ journal_close(j);
+
+ journal_state_init(&state);
+ CHECK(journal_restore(dir, id, &state) == 0 &&
+ state.have_epoch && state.epoch.reason == SYSTEM_EPOCH_COMPACT &&
+ state.nartifacts == 1 &&
+ strcmp(state.artifacts[0].id, artifact_id) == 0 &&
+ state.artifacts[0].retained_bytes == 12345);
+ journal_state_free(&state);
+
+ j = journal_open(dir, id);
+ CHECK(j != NULL &&
+ journal_system_epoch(j, &clear, NULL, NULL, 0) == 0);
+ journal_close(j);
+ journal_state_init(&state);
+ CHECK(journal_restore(dir, id, &state) == 0 &&
+ state.have_epoch && state.epoch.reason == SYSTEM_EPOCH_CLEAR &&
+ count(&state.messages) == 0 && state.nartifacts == 1 &&
+ strcmp(state.artifacts[0].id, artifact_id) == 0 &&
+ state.artifacts[0].retained_bytes == 12345);
+ journal_state_free(&state);
+
+ system_epoch_free(&clear);
+ system_epoch_free(&compact);
+ system_epoch_free(&start);
+}
+
+static void
test_roundtrip(void)
{
struct journal *j;
test_roundtrip();
test_system_epoch_roundtrip();
test_epoch_retained_transactions();
+ test_artifact_survives_epoch_resets();
test_epoch_reasons_and_legacy();
test_epoch_malformed_and_bounds();
test_resume_fidelity();
blob - 1096a8e195db44bbff50609519907374c52cbb0d
blob + 4a6a0b39bb69dddf944dcf9d6b66b9cfdd438de2
--- regress/openai/run.sh
+++ regress/openai/run.sh
lx=$dir/libexec/fugu
mkdir -p "$lx" "$dir/home"
-for w in fugu-tty fugu-api fugu-web fugu-tool fugu-editor; do
+for w in fugu-tty fugu-api fugu-web fugu-tool fugu-artifact fugu-editor; do
b=$root/src/$w/obj/$w
[ -x "$b" ] || b=$root/src/$w/$w
[ -x "$b" ] || { echo "FAIL - $w not built"; exit 1; }
blob - 75a7e52480e04e44c8e98f6876b9d9a109855c56
blob + f7eba65eae05bc013ec6f6c18228eef209360fa7
--- regress/print/run.sh
+++ regress/print/run.sh
lx=$dir/libexec/fugu
mkdir -p "$lx"
-for worker in fugu-tty fugu-api fugu-web fugu-tool fugu-editor; do
+for worker in fugu-tty fugu-api fugu-web fugu-tool fugu-artifact fugu-editor; do
bin=$root/src/$worker/obj/$worker
[ -x "$bin" ] || bin=$root/src/$worker/$worker
[ -x "$bin" ] || { echo "FAIL - $worker not built"; exit 1; }
blob - 4b9b9728466d28a8ebfb55104ce6689584699477
blob + 0c97435af5abb2ff9df5621fc131d3b2ae65447a
--- regress/sandbox/Makefile
+++ regress/sandbox/Makefile
PROG= harness
-SRCS= harness.c priv.c log.c xmalloc.c
+SRCS= harness.c priv.c artifact_lock.c artifact_pipe.c log.c xmalloc.c
NOMAN= yes
.PATH: ${.CURDIR}/../../src/fugu ${.CURDIR}/../../src/common
blob - fbde8b9464fceb0681f58f70fde07b85dd63e73e
blob + 2b83b3a84f03afd1881af57da7905d73d2712fa4
--- regress/sandbox/harness.c
+++ regress/sandbox/harness.c
*/
#include <sys/types.h>
+#include <sys/stat.h>
#include <sys/wait.h>
#include <err.h>
#include <errno.h>
+#include <fcntl.h>
#include <imsg.h>
#include <poll.h>
#include <signal.h>
#include "log.h"
#include "priv.h"
#include "proto.h"
+#include "artifact_lock.h"
+#include "artifact_pipe.h"
#define TEST_BAD_HELLO (-1)
#define TEST_BAD_HELLO_ID (-2)
#define TEST_BAD_PING (-3)
#define TEST_BAD_PING_ID (-4)
#define TEST_CLAUDE_MODELS (-5)
+#define TEST_TOOL_DESCRIPTOR_READ (-6)
+#define TEST_TOOL_SHELL_NO_STREAM (-7)
+#define TEST_TOOL_REORDERED_STREAM (-8)
+#define TEST_TOOL_SHELL (-9)
+#define TEST_TOOL_FREEZE_SWAP (-10)
+#define TEST_TOOL_FREEZE_TERMINAL (-11)
+#define TEST_TOOL_FREEZE_UNAVAILABLE (-12)
+#define TEST_TOOL_FREEZE_CANCEL (-13)
+static const char artifact_id_one[] =
+ "0123456789abcdef0123456789abcdef";
+static const char artifact_id_two[] =
+ "fedcba9876543210fedcba9876543210";
+
/* Blocking imsg read of one message of the expected type. */
static int
expect_msg(struct imsgbuf *ibuf, uint32_t want, struct imsg *out)
return (-1);
if (n != 0) {
if (imsg_get_type(out) != want) {
+ warnx("expected imsg %u, got %u", want,
+ imsg_get_type(out));
imsg_free(out);
return (-1);
}
}
pfd.fd = ibuf->fd;
pfd.events = POLLIN;
- if (poll(&pfd, 1, 5000) <= 0)
+ if (poll(&pfd, 1, 5000) <= 0) {
+ warnx("timed out waiting for imsg %u", want);
return (-1); /* worker died or timed out */
- if ((n = imsgbuf_read(ibuf)) == -1 || n == 0)
+ }
+ if ((n = imsgbuf_read(ibuf)) == -1 || n == 0) {
+ warnx("worker closed while waiting for imsg %u", want);
return (-1);
+ }
}
}
send_msg_id(ibuf, type, 0, data, len);
}
+static int
+artifact_id_valid(const char id[FUGU_ARTIFACT_ID_BUFSZ])
+{
+ size_t i;
+
+ for (i = 0; i < FUGU_ARTIFACT_ID_LEN; i++)
+ if (!((id[i] >= '0' && id[i] <= '9') ||
+ (id[i] >= 'a' && id[i] <= 'f')))
+ return (0);
+ return (id[FUGU_ARTIFACT_ID_LEN] == '\0');
+}
+
+/* Exercise the real post-lockdown SCM_RIGHTS stream exchange. */
+static void
+exercise_artifact_stream(struct imsgbuf *ibuf)
+{
+ static const char probe[] = "artifact-pipe-probe";
+ struct artifact_turn_ack ack;
+ struct artifact_begin begin;
+ struct artifact_stream stream;
+ struct artifact_terminal terminal;
+ struct imsg imsg;
+ int fd, fdflags;
+
+ send_msg(ibuf, FUGU_IMSG_ARTIFACT_TURN_BEGIN, NULL, 0);
+ if (expect_msg(ibuf, FUGU_IMSG_ARTIFACT_TURN_ACK, &imsg) == -1 ||
+ imsg_get_len(&imsg) != sizeof(ack) ||
+ imsg_get_id(&imsg) != 0 ||
+ imsg_get_data(&imsg, &ack, sizeof(ack)) == -1 ||
+ ack.control != FUGU_IMSG_ARTIFACT_TURN_BEGIN)
+ errx(2, "fugu-artifact sent malformed Turn begin ACK");
+ imsg_free(&imsg);
+
+ memset(&begin, 0, sizeof(begin));
+ begin.kind = ARTIFACT_BEGIN_FOREGROUND;
+ send_msg_id(ibuf, FUGU_IMSG_ARTIFACT_BEGIN, 1,
+ &begin, sizeof(begin));
+ if (expect_msg(ibuf, FUGU_IMSG_ARTIFACT_STREAM, &imsg) == -1 ||
+ imsg_get_len(&imsg) != sizeof(stream) ||
+ imsg_get_id(&imsg) != 1 ||
+ imsg_get_data(&imsg, &stream, sizeof(stream)) == -1 ||
+ stream.status != ARTIFACT_STREAM_AVAILABLE ||
+ !artifact_id_valid(stream.id) ||
+ (fd = imsg_get_fd(&imsg)) == -1 ||
+ (fdflags = fcntl(fd, F_GETFD)) == -1 ||
+ fcntl(fd, F_SETFD, fdflags | FD_CLOEXEC) == -1 ||
+ !artifact_producer_endpoint_valid(fd))
+ errx(2, "fugu-artifact sent malformed producer endpoint");
+ imsg_free(&imsg);
+ if (write(fd, probe, sizeof(probe) - 1) !=
+ (ssize_t)(sizeof(probe) - 1))
+ errx(2, "write Artifact producer endpoint");
+ close(fd);
+
+ if (expect_msg(ibuf, FUGU_IMSG_ARTIFACT_TERMINAL, &imsg) == -1 ||
+ imsg_get_len(&imsg) != sizeof(terminal) ||
+ imsg_get_id(&imsg) != 1 ||
+ imsg_get_data(&imsg, &terminal, sizeof(terminal)) == -1 ||
+ terminal.status != ARTIFACT_STORE_OK ||
+ terminal.retained_bytes != sizeof(probe) - 1 ||
+ memcmp(terminal.id, stream.id, sizeof(stream.id)) != 0)
+ errx(2, "fugu-artifact sent malformed stream terminal");
+ imsg_free(&imsg);
+
+ send_msg(ibuf, FUGU_IMSG_ARTIFACT_TURN_ABORT, NULL, 0);
+ if (expect_msg(ibuf, FUGU_IMSG_ARTIFACT_TURN_ACK, &imsg) == -1 ||
+ imsg_get_len(&imsg) != sizeof(ack) ||
+ imsg_get_id(&imsg) != 0 ||
+ imsg_get_data(&imsg, &ack, sizeof(ack)) == -1 ||
+ ack.control != FUGU_IMSG_ARTIFACT_TURN_ABORT)
+ errx(2, "fugu-artifact sent malformed Turn abort ACK");
+ imsg_free(&imsg);
+}
+
+static void
+send_msg_fd_id(struct imsgbuf *ibuf, uint32_t type, uint32_t id, int fd,
+ const void *data, size_t len)
+{
+ if (imsg_compose(ibuf, type, id, -1, fd, data, len) == -1)
+ errx(2, "descriptor imsg_compose");
+ if (imsgbuf_flush(ibuf) == -1)
+ errx(2, "descriptor imsgbuf_flush");
+}
+
+static int
+send_artifact_stream_status(struct imsgbuf *ibuf, uint32_t id,
+ const char *artifact_id, int available)
+{
+ struct artifact_stream stream;
+ int p[2];
+
+ memset(&stream, 0, sizeof(stream));
+ stream.status = available ? ARTIFACT_STREAM_AVAILABLE :
+ ARTIFACT_STREAM_UNAVAILABLE;
+ if (!available) {
+ send_msg_id(ibuf, FUGU_IMSG_TOOL_ARTIFACT_STREAM, id,
+ &stream, sizeof(stream));
+ return (-1);
+ }
+ if (pipe2(p, O_CLOEXEC) == -1)
+ errx(2, "Artifact fixture pipe");
+ strlcpy(stream.id, artifact_id, sizeof(stream.id));
+ send_msg_fd_id(ibuf, FUGU_IMSG_TOOL_ARTIFACT_STREAM, id, p[1],
+ &stream, sizeof(stream));
+ return (p[0]);
+}
+
+static int
+send_artifact_stream(struct imsgbuf *ibuf, uint32_t id)
+{
+ return (send_artifact_stream_status(ibuf, id, artifact_id_one, 1));
+}
+
+static void
+send_tool_req(struct imsgbuf *ibuf, uint32_t id, const char *name)
+{
+ struct tool_req req;
+
+ memset(&req, 0, sizeof(req));
+ strlcpy(req.name, name, sizeof(req.name));
+ send_msg_id(ibuf, FUGU_IMSG_TOOL_REQ, id, &req, sizeof(req));
+}
+
+static void
+send_tool_call(struct imsgbuf *ibuf, uint32_t id, const char *name,
+ const char *arg)
+{
+ send_tool_req(ibuf, id, name);
+ if (arg != NULL)
+ send_msg_id(ibuf, FUGU_IMSG_TOOL_ARG, id, arg, strlen(arg));
+ send_msg_id(ibuf, FUGU_IMSG_TOOL_RUN, id, NULL, 0);
+}
+
+static int
+tool_turn_control(struct imsgbuf *ibuf, uint32_t control)
+{
+ struct artifact_turn_ack ack;
+ struct imsg imsg;
+ int ok;
+
+ send_msg(ibuf, control, NULL, 0);
+ if (expect_msg(ibuf, FUGU_IMSG_TOOL_TURN_ACK, &imsg) == -1)
+ return (-1);
+ ok = imsg_get_id(&imsg) == 0 &&
+ imsg_get_len(&imsg) == sizeof(ack) &&
+ imsg_get_data(&imsg, &ack, sizeof(ack)) == 0 &&
+ ack.control == control;
+ imsg_free(&imsg);
+ return (ok ? 0 : -1);
+}
+
+static int
+expect_tool_success(struct imsgbuf *ibuf, uint32_t id, int artifact)
+{
+ struct tool_result result;
+ struct imsg imsg;
+ u_char *p;
+ size_t len;
+ int ok;
+
+ if (expect_msg(ibuf, FUGU_IMSG_TOOL_OUT, &imsg) == -1 ||
+ imsg_get_id(&imsg) != id)
+ return (-1);
+ len = imsg_get_len(&imsg);
+ p = len == 0 ? NULL : malloc(len);
+ if (len != 0 && (p == NULL || imsg_get_data(&imsg, p, len) == -1)) {
+ free(p);
+ imsg_free(&imsg);
+ return (-1);
+ }
+ ok = p != NULL && memmem(p, len, "scope-ok", 8) != NULL;
+ free(p);
+ imsg_free(&imsg);
+ if (!ok || expect_msg(ibuf, FUGU_IMSG_TOOL_RESULT, &imsg) == -1)
+ return (-1);
+ if (imsg_get_id(&imsg) != id || imsg_get_len(&imsg) != sizeof(result) ||
+ imsg_get_data(&imsg, &result, sizeof(result)) == -1) {
+ imsg_free(&imsg);
+ return (-1);
+ }
+ imsg_free(&imsg);
+ return (!result.is_error && !!result.artifact == !!artifact &&
+ !result.spool_frozen ? 0 : -1);
+}
+
+/* A managed shell_output publishes its ownership frame before ordinary
+ * output. want_adopted == -1 proves that no frame preceded the result. */
+static int
+expect_tool_artifact_result(struct imsgbuf *ibuf, uint32_t id,
+ int want_adopted, int want_detach, uint32_t job_id, const char *frozen_id,
+ const char *replacement_id, struct tool_result *result)
+{
+ static const char zero[FUGU_ARTIFACT_ID_BUFSZ];
+ struct artifact_spool_rotate rotate;
+ struct imsg imsg;
+ const char *want_frozen = frozen_id == NULL ? zero : frozen_id;
+ const char *want_replacement = replacement_id == NULL ? zero :
+ replacement_id;
+ int ok;
+
+ if (want_adopted != -1) {
+ if (expect_msg(ibuf, FUGU_IMSG_TOOL_ARTIFACT_ROTATE, &imsg) == -1)
+ return (-1);
+ ok = imsg_get_id(&imsg) == id &&
+ imsg_get_len(&imsg) == sizeof(rotate) &&
+ imsg_get_data(&imsg, &rotate, sizeof(rotate)) == 0 &&
+ rotate.job_id == job_id &&
+ rotate.replacement_adopted == (uint32_t)want_adopted &&
+ rotate.detach == (uint32_t)want_detach &&
+ memcmp(rotate.frozen_id, want_frozen,
+ sizeof(rotate.frozen_id)) == 0 &&
+ memcmp(rotate.replacement_id, want_replacement,
+ sizeof(rotate.replacement_id)) == 0;
+ imsg_free(&imsg);
+ if (!ok) {
+ warnx("wrong freeze frame: job=%u adopted=%u detach=%u "
+ "frozen=%s replacement=%s", rotate.job_id,
+ rotate.replacement_adopted, rotate.detach,
+ rotate.frozen_id, rotate.replacement_id);
+ return (-1);
+ }
+ }
+ if (expect_msg(ibuf, FUGU_IMSG_TOOL_OUT, &imsg) == -1)
+ return (-1);
+ ok = imsg_get_id(&imsg) == id;
+ imsg_free(&imsg);
+ if (!ok || expect_msg(ibuf, FUGU_IMSG_TOOL_RESULT, &imsg) == -1)
+ return (-1);
+ memset(result, 0, sizeof(*result));
+ ok = imsg_get_id(&imsg) == id &&
+ imsg_get_len(&imsg) == sizeof(*result) &&
+ imsg_get_data(&imsg, result, sizeof(*result)) == 0;
+ imsg_free(&imsg);
+ return (ok ? 0 : -1);
+}
+
+static int
+send_artifact_cancel(struct imsgbuf *ibuf, uint32_t job_id, const char *id)
+{
+ struct artifact_job_cancel_ack ack;
+ struct artifact_record record;
+ struct imsg imsg;
+ int ok;
+
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, id, sizeof(record.id));
+ send_msg_id(ibuf, FUGU_IMSG_TOOL_ARTIFACT_CANCEL, job_id, &record,
+ sizeof(record));
+ if (expect_msg(ibuf, FUGU_IMSG_TOOL_ARTIFACT_CANCEL_ACK, &imsg) == -1)
+ return (-1);
+ ok = imsg_get_id(&imsg) == job_id &&
+ imsg_get_len(&imsg) == sizeof(ack) &&
+ imsg_get_data(&imsg, &ack, sizeof(ack)) == 0 &&
+ ack.job_id == job_id && strcmp(ack.artifact_id, id) == 0;
+ imsg_free(&imsg);
+ return (ok ? 0 : -1);
+}
+
+static int
+test_tool_freeze_frames(struct imsgbuf *ibuf, pid_t worker_pid, int action)
+{
+ static const char zero[FUGU_ARTIFACT_ID_BUFSZ];
+ struct artifact_record record;
+ struct tool_result result;
+ char arg[64];
+ const char *command;
+ pid_t notifier = -1;
+ uint32_t bg_id = 101, poll_id = 102, job_id;
+ int old_fd = -1, replacement_fd = -1, status;
+ int old_available = action != TEST_TOOL_FREEZE_UNAVAILABLE;
+ int rc = -1;
+
+ command = action == TEST_TOOL_FREEZE_CANCEL ?
+ "{\"command\":\"yes x\"}" :
+ action == TEST_TOOL_FREEZE_SWAP ?
+ "{\"command\":\"printf swap; sleep 30\"}" :
+ "{\"command\":\"/usr/bin/true\"}";
+ if (tool_turn_control(ibuf, FUGU_IMSG_TOOL_TURN_BEGIN) == -1)
+ goto done;
+ old_fd = send_artifact_stream_status(ibuf, bg_id, artifact_id_one,
+ old_available);
+ send_tool_call(ibuf, bg_id, "shell_bg", command);
+ if (expect_tool_artifact_result(ibuf, bg_id, -1, 0, 0, NULL, NULL,
+ &result) == -1 || result.is_error || result.spool_frozen ||
+ !result.spool_adopted ||
+ result.background_job_id <= 0) {
+ warnx("wrong shell_bg result: error=%d frozen=%d adopted=%d job=%d",
+ result.is_error, result.spool_frozen, result.spool_adopted,
+ result.background_job_id);
+ goto done;
+ }
+ job_id = (uint32_t)result.background_job_id;
+ if (tool_turn_control(ibuf, FUGU_IMSG_TOOL_TURN_COMMIT) == -1)
+ goto done;
+ usleep(action == TEST_TOOL_FREEZE_CANCEL ? 700000 : 350000);
+ if (tool_turn_control(ibuf, FUGU_IMSG_TOOL_TURN_BEGIN) == -1)
+ goto done;
+ replacement_fd = send_artifact_stream_status(ibuf, poll_id,
+ artifact_id_two, 1);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%u}", job_id);
+ if (action == TEST_TOOL_FREEZE_CANCEL) {
+ notifier = fork();
+ if (notifier == -1)
+ goto done;
+ if (notifier == 0) {
+ usleep(200000);
+ (void)kill(worker_pid, SIGUSR1);
+ _exit(0);
+ }
+ }
+ send_tool_call(ibuf, poll_id, "shell_output", arg);
+ if (action == TEST_TOOL_FREEZE_SWAP) {
+ if (expect_tool_artifact_result(ibuf, poll_id, 1, 1, job_id,
+ artifact_id_one, artifact_id_two, &result) == -1 ||
+ result.is_error || !result.spool_frozen ||
+ !result.spool_adopted ||
+ result.background_job_id != (int)job_id) {
+ warnx("wrong swap result: error=%d frozen=%d adopted=%d job=%d",
+ result.is_error, result.spool_frozen,
+ result.spool_adopted, result.background_job_id);
+ goto done;
+ }
+ } else if (action == TEST_TOOL_FREEZE_TERMINAL) {
+ if (expect_tool_artifact_result(ibuf, poll_id, 0, 0, job_id,
+ artifact_id_one, NULL, &result) == -1 || result.is_error ||
+ !result.spool_frozen || result.spool_adopted ||
+ result.background_job_id != (int)job_id ||
+ strcmp(result.artifact_id, artifact_id_one) != 0) {
+ warnx("wrong available terminal result");
+ goto done;
+ }
+ } else if (action == TEST_TOOL_FREEZE_UNAVAILABLE) {
+ if (expect_tool_artifact_result(ibuf, poll_id, 0, 0, job_id,
+ NULL, NULL, &result) == -1 || result.is_error ||
+ !result.spool_frozen || result.spool_adopted ||
+ result.background_job_id != (int)job_id ||
+ memcmp(result.artifact_id, zero,
+ sizeof(result.artifact_id)) != 0) {
+ warnx("wrong unavailable terminal result");
+ goto done;
+ }
+ } else {
+ if (expect_tool_artifact_result(ibuf, poll_id, -1, 0, job_id,
+ NULL, NULL, &result) == -1 || result.is_error ||
+ result.spool_frozen || result.spool_adopted ||
+ result.background_job_id != (int)job_id) {
+ warnx("wrong cancelled pre-freeze result");
+ goto done;
+ }
+ }
+ if (notifier != -1) {
+ if (waitpid(notifier, &status, 0) == -1 || !WIFEXITED(status) ||
+ WEXITSTATUS(status) != 0)
+ goto done;
+ notifier = -1;
+ }
+ if (action == TEST_TOOL_FREEZE_TERMINAL) {
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, artifact_id_one, sizeof(record.id));
+ send_msg_id(ibuf, FUGU_IMSG_TOOL_ARTIFACT_RELEASE, job_id,
+ &record, sizeof(record));
+ }
+ if (tool_turn_control(ibuf, FUGU_IMSG_TOOL_TURN_ABORT) == -1)
+ goto done;
+ if (action == TEST_TOOL_FREEZE_SWAP) {
+ if (send_artifact_cancel(ibuf, job_id, artifact_id_two) == -1)
+ goto done;
+ } else if (action == TEST_TOOL_FREEZE_CANCEL) {
+ if (send_artifact_cancel(ibuf, job_id, artifact_id_one) == -1)
+ goto done;
+ }
+ rc = 0;
+done:
+ if (notifier > 0) {
+ (void)kill(notifier, SIGTERM);
+ (void)waitpid(notifier, NULL, 0);
+ }
+ if (old_fd != -1)
+ close(old_fd);
+ if (replacement_fd != -1)
+ close(replacement_fd);
+ return (rc);
+}
+
+static int
+worker_rejected(pid_t pid)
+{
+ int i, status;
+ pid_t rc;
+
+ for (i = 0; i < 200; i++) {
+ rc = waitpid(pid, &status, WNOHANG);
+ if (rc == pid)
+ return (WIFEXITED(status) && WEXITSTATUS(status) == 1);
+ if (rc == -1)
+ return (0);
+ usleep(10000);
+ }
+ (void)kill(pid, SIGKILL);
+ (void)waitpid(pid, &status, 0);
+ return (0);
+}
+
static enum fugu_role
role_of(const char *s)
{
return (ROLE_WEB);
if (strcmp(s, "fugu-tool") == 0)
return (ROLE_TOOL);
+ if (strcmp(s, "fugu-artifact") == 0)
+ return (ROLE_ARTIFACT);
if (strcmp(s, "fugu-editor") == 0)
return (ROLE_EDIT);
errx(1, "unknown role %s", s);
return (TEST_BAD_PING_ID);
if (strcmp(s, "claude_models") == 0)
return (TEST_CLAUDE_MODELS);
+ if (strcmp(s, "tool_descriptor_read") == 0)
+ return (TEST_TOOL_DESCRIPTOR_READ);
+ if (strcmp(s, "tool_shell_no_stream") == 0)
+ return (TEST_TOOL_SHELL_NO_STREAM);
+ if (strcmp(s, "tool_reordered_stream") == 0)
+ return (TEST_TOOL_REORDERED_STREAM);
+ if (strcmp(s, "tool_shell") == 0)
+ return (TEST_TOOL_SHELL);
+ if (strcmp(s, "tool_freeze_swap") == 0)
+ return (TEST_TOOL_FREEZE_SWAP);
+ if (strcmp(s, "tool_freeze_terminal") == 0)
+ return (TEST_TOOL_FREEZE_TERMINAL);
+ if (strcmp(s, "tool_freeze_unavailable") == 0)
+ return (TEST_TOOL_FREEZE_UNAVAILABLE);
+ if (strcmp(s, "tool_freeze_cancel") == 0)
+ return (TEST_TOOL_FREEZE_CANCEL);
if (strcmp(s, "none") == 0)
return (0);
errx(1, "unknown action %s", s);
errx(2, "spawn %s", rolename);
if (imsgbuf_init(&ibuf, kid.fd) == -1)
errx(2, "imsgbuf_init");
+ imsgbuf_allow_fdpass(&ibuf);
if (action == TEST_BAD_HELLO || action == TEST_BAD_HELLO_ID) {
if (action == TEST_BAD_HELLO)
send_msg(&ibuf, FUGU_IMSG_HELLO, &byte, sizeof(byte));
memset(&h, 0, sizeof(h));
h.allow_write = !nowrite;
h.allow_subprocess_net = 0;
+ h.max_bg_jobs = 4;
send_msg(&ibuf, FUGU_IMSG_HELLO, &h, sizeof(h));
} else if (role == ROLE_WEB) {
struct hello_web h;
memset(&h, 0, sizeof(h));
strlcpy(h.search_provider, "kagi", sizeof(h.search_provider));
send_msg(&ibuf, FUGU_IMSG_HELLO, &h, sizeof(h));
+ } else if (role == ROLE_ARTIFACT) {
+ struct hello_artifact h;
+ int fdflags, lock_fd;
+
+ memset(&h, 0, sizeof(h));
+ h.fresh = 1;
+ if (snprintf(h.session, sizeof(h.session), "sandbox-%ld",
+ (long)getpid()) < 0 ||
+ getcwd(h.root, sizeof(h.root)) == NULL ||
+ strlcat(h.root, "/artifacts", sizeof(h.root)) >=
+ sizeof(h.root))
+ errx(2, "Artifact sandbox path");
+ send_msg(&ibuf, FUGU_IMSG_HELLO, &h, sizeof(h));
+
+ /* The lifetime lock is the Artifact Role's exact pre-READY
+ * descriptor exchange. Consume and validate it before the common
+ * worker READY message, just as the Coordinator does. */
+ if (expect_msg(&ibuf, FUGU_IMSG_ARTIFACT_LIFETIME_LOCK,
+ &imsg) == -1)
+ errx(2, "fugu-artifact omitted lifetime lock");
+ lock_fd = imsg_get_fd(&imsg);
+ if (imsg_get_len(&imsg) != 0 || imsg_get_id(&imsg) != 0 ||
+ lock_fd == -1 ||
+ (fdflags = fcntl(lock_fd, F_GETFD)) == -1 ||
+ fcntl(lock_fd, F_SETFD, fdflags | FD_CLOEXEC) == -1 ||
+ !artifact_lifetime_lock_valid(lock_fd, h.root, h.session))
+ errx(2, "fugu-artifact sent malformed lifetime lock");
+ close(lock_fd);
+ imsg_free(&imsg);
} else {
send_msg(&ibuf, FUGU_IMSG_HELLO, NULL, 0);
}
if (imsg_get_len(&imsg) != 0 || imsg_get_id(&imsg) != 0)
errx(2, "%s sent malformed READY", rolename);
imsg_free(&imsg);
+ if (role == ROLE_ARTIFACT) {
+ /* Mirror the Coordinator's explicit startup phase boundary before
+ * exercising any steady-state Turn or selftest request. */
+ send_msg(&ibuf, FUGU_IMSG_ARTIFACT_RECONCILE_END, NULL, 0);
+ if (expect_msg(&ibuf, FUGU_IMSG_ARTIFACT_RECONCILE_ACK,
+ &imsg) == -1 || imsg_get_len(&imsg) != 0 ||
+ imsg_get_id(&imsg) != 0 || imsg_get_fd(&imsg) != -1)
+ errx(2, "fugu-artifact did not finish reconciliation");
+ imsg_free(&imsg);
+ }
+ if (role == ROLE_TOOL && (action == TEST_TOOL_FREEZE_SWAP ||
+ action == TEST_TOOL_FREEZE_TERMINAL ||
+ action == TEST_TOOL_FREEZE_UNAVAILABLE ||
+ action == TEST_TOOL_FREEZE_CANCEL)) {
+ int ok = test_tool_freeze_frames(&ibuf, kid.pid, action) == 0;
+
+ if (!ok) {
+ (void)kill(kid.pid, SIGKILL);
+ (void)waitpid(kid.pid, &status, 0);
+ return (1);
+ }
+ send_msg(&ibuf, FUGU_IMSG_SHUTDOWN, NULL, 0);
+ if (waitpid(kid.pid, &status, 0) == -1)
+ errx(2, "waitpid Tool freeze frame");
+ return (WIFEXITED(status) && WEXITSTATUS(status) == 0 ? 0 : 1);
+ }
+
+ if (role == ROLE_TOOL && (action == TEST_TOOL_DESCRIPTOR_READ ||
+ action == TEST_TOOL_SHELL_NO_STREAM ||
+ action == TEST_TOOL_REORDERED_STREAM ||
+ action == TEST_TOOL_SHELL)) {
+ static const char shell_arg[] =
+ "{\"command\":\"printf scope-ok\"}";
+ uint32_t id = 77;
+ int artifact_fd = -1, ok;
+
+ if (action == TEST_TOOL_DESCRIPTOR_READ) {
+ artifact_fd = send_artifact_stream(&ibuf, id);
+ send_tool_req(&ibuf, id, "read");
+ } else if (action == TEST_TOOL_SHELL_NO_STREAM) {
+ send_tool_req(&ibuf, 0, "shell");
+ } else if (action == TEST_TOOL_REORDERED_STREAM) {
+ send_tool_req(&ibuf, 0, "read");
+ artifact_fd = send_artifact_stream(&ibuf, id);
+ } else {
+ if (readonly)
+ id = 0;
+ else
+ artifact_fd = send_artifact_stream(&ibuf, id);
+ send_tool_req(&ibuf, id, "shell");
+ send_msg_id(&ibuf, FUGU_IMSG_TOOL_ARG, id, shell_arg,
+ sizeof(shell_arg) - 1);
+ send_msg_id(&ibuf, FUGU_IMSG_TOOL_RUN, id, NULL, 0);
+ ok = expect_tool_success(&ibuf, id, !readonly) == 0;
+ if (artifact_fd != -1)
+ close(artifact_fd);
+ if (!ok) {
+ (void)kill(kid.pid, SIGKILL);
+ (void)waitpid(kid.pid, &status, 0);
+ return (1);
+ }
+ send_msg(&ibuf, FUGU_IMSG_SHUTDOWN, NULL, 0);
+ if (waitpid(kid.pid, &status, 0) == -1)
+ errx(2, "waitpid Tool scope success");
+ return (WIFEXITED(status) && WEXITSTATUS(status) == 0 ?
+ 0 : 1);
+ }
+ ok = worker_rejected(kid.pid);
+ if (artifact_fd != -1)
+ close(artifact_fd);
+ return (ok ? 0 : 1);
+ }
+
if (action == TEST_CLAUDE_MODELS) {
struct model_ent me;
u_char p[sizeof(me) + 1 + FUGU_MODEL_ID_MAX];
}
if (action == 0) {
- /* handshake-only: verify liveness then shut down cleanly */
+ /* Verify liveness and Artifact's typed descriptor exchange. */
+ if (role == ROLE_ARTIFACT)
+ exercise_artifact_stream(&ibuf);
send_msg(&ibuf, FUGU_IMSG_PING, NULL, 0);
if (expect_msg(&ibuf, FUGU_IMSG_PONG, &imsg) == -1)
errx(2, "%s did not pong", rolename);
blob - dfeb858e0b2c2973fb5f97752d25b6efcb7d9212
blob + 1df4955f3793f707c20254cf3f3a0fa5f047343a
--- regress/sandbox/run.sh
+++ regress/sandbox/run.sh
trap 'rm -rf "$dir"' EXIT INT TERM
# Locate and stage the real worker binaries.
-for w in fugu-tty fugu-api fugu-web fugu-tool fugu-editor; do
+for w in fugu-tty fugu-api fugu-web fugu-tool fugu-artifact fugu-editor; do
bin=$root/src/$w/obj/$w
[ -x "$bin" ] || bin=$root/src/$w/$w
if [ ! -x "$bin" ]; then
run_case fugu-api claude_models ok -
run_case fugu-web none ok -
run_case fugu-tool none ok -
+run_case fugu-artifact none ok -
run_case fugu-editor none ok -
+# Persistent Lead Artifact scope is an exact protocol property: scoped calls
+# require one descriptor, unscoped calls forbid it, and a descriptor cannot be
+# injected after a descriptor-free request has begun. The -r executor remains
+# descriptor-free and can run its ordinary inspection shell unchanged.
+run_case fugu-tool tool_descriptor_read reject -
+run_case fugu-tool tool_shell_no_stream reject -
+run_case fugu-tool tool_reordered_stream reject -
+run_case fugu-tool tool_shell ok -
+run_case fugu-tool-r tool_shell ok -
+
+# shell_output's out-of-band frame is the ownership proof: a running swap
+# names both ids, terminal freezes name no replacement (including a zero-id
+# unavailable old spool), and an interrupted pre-freeze call emits no frame.
+run_case fugu-tool tool_freeze_swap ok -
+run_case fugu-tool tool_freeze_terminal ok -
+run_case fugu-tool tool_freeze_unavailable ok -
+run_case fugu-tool tool_freeze_cancel ok -
+
# I10/I13: startup and liveness frames are exact typed exchanges. The TTY
# and editor historically accepted malformed HELLO payloads; the common
# worker runtime must also reject nonzero control correlation IDs and malformed
run_case fugu-web exec dead execve
run_case fugu-web open_etc dead open
+# The Artifact Role can use only its private managed root: it cannot execute,
+# reach the network, read configuration, access the tty, or write the project.
+# The first two are pledge kills; the path probes are unveil-shaped ENOENTs.
+run_case fugu-artifact exec dead execve
+run_case fugu-artifact connect dead socket
+run_case fugu-artifact open_etc errno -
+run_case fugu-artifact open_tty errno -
+run_case fugu-artifact write_intree errno -
+
# I7: the blast room cannot reach the tty.
run_case fugu-web open_tty dead open
blob - 8419f1e7f89a5ed2419fc81ac4e8e053a50195c3
blob + c9cb30793f43409f0341a3a161f935ea4d158cda
--- regress/tools/Makefile
+++ regress/tools/Makefile
NOMAN= yes
CFLAGS+= -I${.CURDIR}/../../src/fugu -I${.CURDIR}/../../src/fugu-tool
+CFLAGS+= -DJOB_TESTING
.PATH: ${.CURDIR}/../../src/fugu-tool ${.CURDIR}/../../src/fugu
REGRESS_TARGETS= run-tools
blob - e3cb883d0b5c9c2820c1174582e015b39cb195ee
blob + bf5cedd91dd6f600a031196206f2f8eb9f4312eb
--- regress/tools/tools_test.c
+++ regress/tools/tools_test.c
#include <stdlib.h>
#include <string.h>
#include <syslog.h>
+#include <time.h>
#include <unistd.h>
#include "log.h"
static struct tool_ctx *ctx;
static struct buf out;
static int err_flag;
+static uint32_t managed_correlation = 100;
+static const char artifact_ids[][ARTIFACT_ID_HEX_LEN + 1] = {
+ "10000000000000000000000000000001",
+ "20000000000000000000000000000002",
+ "30000000000000000000000000000003",
+ "40000000000000000000000000000004",
+ "50000000000000000000000000000005",
+ "60000000000000000000000000000006",
+ "70000000000000000000000000000007",
+ "80000000000000000000000000000008"
+};
+
static void
run(const char *name, const char *input)
{
tool_exec(ctx, name, input, strlen(input), &out, &err_flag);
}
+static void
+managed_run(const char *name, const char *input, const char *id, int fd,
+ int available)
+{
+ ctx->artifact_mode = 1;
+ ctx->artifact_available = available;
+ ctx->artifact_fd = fd;
+ ctx->artifact_id = available ? id : NULL;
+ ctx->artifact_correlation = managed_correlation++;
+ run(name, input);
+}
+
+static void
+cancel_active_signal(int sig)
+{
+ (void)sig;
+ tool_cancel_active();
+}
+
+static void
+nonblock_fd(int fd)
+{
+ int flags = fcntl(fd, F_GETFL);
+
+ if (flags == -1 || fcntl(fd, F_SETFL, flags | O_NONBLOCK) == -1)
+ err(1, "nonblocking fixture fd");
+}
+
+static uint64_t
+drain_bytes(int fd)
+{
+ u_char bytes[64 * 1024];
+ uint64_t total = 0;
+ ssize_t n;
+
+ for (;;) {
+ n = read(fd, bytes, sizeof(bytes));
+ if (n > 0) {
+ total += (uint64_t)n;
+ continue;
+ }
+ if (n == -1 && errno == EINTR)
+ continue;
+ break;
+ }
+ return (total);
+}
+
+static void
+fill_pipe(int fd)
+{
+ u_char bytes[4096];
+ ssize_t n;
+
+ memset(bytes, 0xa5, sizeof(bytes));
+ nonblock_fd(fd);
+ for (;;) {
+ n = write(fd, bytes, sizeof(bytes));
+ if (n > 0)
+ continue;
+ if (n == -1 && errno == EINTR)
+ continue;
+ if (n == -1 && (errno == EAGAIN || errno == EWOULDBLOCK))
+ return;
+ err(1, "fill fixture pipe");
+ }
+}
+
+static long
+elapsed_ms(const struct timespec *start, const struct timespec *end)
+{
+ return ((long)(end->tv_sec - start->tv_sec) * 1000L +
+ (end->tv_nsec - start->tv_nsec) / 1000000L);
+}
+
+static void
+managed_clear(void)
+{
+ if (ctx->artifact_fd != -1)
+ close(ctx->artifact_fd);
+ ctx->artifact_fd = -1;
+ ctx->artifact_mode = 0;
+ ctx->artifact_available = 0;
+ ctx->artifact_id = NULL;
+ ctx->artifact_correlation = 0;
+}
+
/* memmem so NUL-bearing results are searched length-aware */
static int
outhas(const char *s)
close(fd);
}
+static void
+test_managed_artifacts(void)
+{
+ struct job_test_state cap_state;
+ struct timespec start, end;
+ u_char sink[16 * 1024];
+ char arg[64];
+ int p1[2], p2[2], job_id, i, duplicate, nullfd, status;
+ pid_t notifier;
+ uint64_t frozen_expected, snapshot, spooled;
+
+ job_set_cap(ctx->jobs, 4);
+
+ /* Foreground capture observes no more than the one byte needed to prove
+ * a cap crossing. Exact-cap EOF is success; crossing is exactly +1. */
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell", "{\"command\":\"/usr/bin/perl -e "
+ "'print q(x) x 16777216'\"}", artifact_ids[0], nullfd, 1);
+ CHECK(!err_flag && !ctx->artifact_cap_reached &&
+ ctx->artifact_termination == ARTIFACT_TERM_EXIT &&
+ ctx->artifact_observed == ARTIFACT_RAW_MAX &&
+ ctx->artifact_raw_len == ARTIFACT_RAW_MAX);
+ managed_clear();
+
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell", "{\"command\":\"/usr/bin/perl -e "
+ "'print q(x) x 17825792'\"}", artifact_ids[0], nullfd, 1);
+ CHECK(err_flag && ctx->artifact_cap_reached &&
+ ctx->artifact_termination == ARTIFACT_TERM_CAP &&
+ ctx->artifact_observed == ARTIFACT_RAW_MAX + 1ULL &&
+ ctx->artifact_raw_len == ARTIFACT_RAW_MAX);
+ managed_clear();
+
+ /* A replacement created in the spawning Turn remains Turn-owned. */
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p1) == 0);
+ nonblock_fd(p1[0]);
+ managed_run("shell_bg",
+ "{\"command\":\"printf owned-marker; sleep 30\"}",
+ artifact_ids[0], p1[1], 1);
+ job_id = ctx->artifact_job_id;
+ CHECK(!err_flag && ctx->artifact_spool_adopted && job_id > 0);
+ for (i = 0; i < 20; i++) {
+ job_pump(ctx->jobs);
+ (void)drain_bytes(p1[0]);
+ usleep(10000);
+ }
+ CHECK(pipe(p2) == 0);
+ nonblock_fd(p2[0]);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[1], p2[1], 1);
+ CHECK(ctx->artifact_job_id == job_id && ctx->artifact_spool_adopted &&
+ !ctx->artifact_spool_detach && ctx->artifact_result_available &&
+ strcmp(ctx->artifact_result_id, artifact_ids[0]) == 0);
+ CHECK(memmem(ctx->artifact_capture.full.data,
+ ctx->artifact_capture.full.len, "owned-marker", 12) != NULL);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[0]) == 0);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+ close(p1[0]);
+ close(p2[0]);
+ run("shell_output", arg);
+ CHECK(err_flag && outhas("no such job"));
+
+ /* A committed job's later poll replacement detaches before abort. */
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p1) == 0);
+ nonblock_fd(p1[0]);
+ managed_run("shell_bg", "{\"command\":\"printf durable; sleep 30\"}",
+ artifact_ids[2], p1[1], 1);
+ job_id = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p2) == 0);
+ nonblock_fd(p2[0]);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[3], p2[1], 1);
+ CHECK(ctx->artifact_spool_adopted && ctx->artifact_spool_detach &&
+ ctx->artifact_job_id == job_id);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[2]) == 0);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+ close(p1[0]);
+ close(p2[0]);
+ run("shell_output", arg);
+ CHECK(!err_flag && !outhas("no such job"));
+ run("shell_kill", arg);
+ CHECK(!err_flag);
+
+ /* A producer that completed with no bytes is reaped and EOF-closed in the
+ * same poll. It must not report running or adopt an endless replacement. */
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"/usr/bin/true\"}",
+ artifact_ids[6], nullfd, 1);
+ job_id = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ usleep(300000);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[7], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_captured &&
+ ctx->artifact_termination == ARTIFACT_TERM_EXIT &&
+ ctx->artifact_observed == 0 && ctx->artifact_raw_len == 0 &&
+ ctx->artifact_result_available && !ctx->artifact_spool_adopted);
+ managed_clear();
+ run("shell_output", arg);
+ CHECK(err_flag && outhas("no such job"));
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[6]) == 0);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[6]) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, job_id, artifact_ids[6]) == -1);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[7]) == -1);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* shell_output freezes one FIONREAD snapshot. A blocked fast producer
+ * cannot refill the pipe into the segment while that snapshot is drained. */
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"yes snapshot\"}",
+ artifact_ids[4], nullfd, 1);
+ job_id = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ for (i = 0; i < 200 && job_test_output_bytes(ctx->jobs, job_id) == 0;
+ i++)
+ usleep(5000);
+ usleep(50000);
+ snapshot = job_test_output_bytes(ctx->jobs, job_id);
+ CHECK(snapshot > 0 && snapshot <= job_test_pump_per_job_limit());
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[5], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_observed == snapshot &&
+ ctx->artifact_raw_len == snapshot && ctx->artifact_spool_adopted &&
+ ctx->artifact_spool_detach);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[4]) == 0);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+ run("shell_kill", arg);
+ CHECK(!err_flag);
+
+ /* Reservation-unavailable capture stays inline while safe and kills the
+ * producer conspicuously exactly when no terminal form can fit. */
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ managed_run("shell_bg", "{\"command\":\"printf inline; sleep 30\"}",
+ artifact_ids[4], -1, 0);
+ job_id = ctx->artifact_job_id;
+ for (i = 0; i < 10; i++)
+ job_pump(ctx->jobs);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[5], -1, 0);
+ CHECK(!err_flag && !ctx->artifact_result_available &&
+ ctx->artifact_termination == ARTIFACT_TERM_RUNNING);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ managed_run("shell_bg", "{\"command\":\"yes unavailable\"}",
+ artifact_ids[4], -1, 0);
+ job_id = ctx->artifact_job_id;
+ for (i = 0; i < 200 &&
+ !job_test_storage_unavailable(ctx->jobs, job_id); i++) {
+ job_pump(ctx->jobs);
+ usleep(5000);
+ }
+ CHECK(job_test_storage_unavailable(ctx->jobs, job_id));
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[5], -1, 0);
+ CHECK(err_flag &&
+ ctx->artifact_termination == ARTIFACT_TERM_STORAGE_UNAVAILABLE &&
+ ctx->artifact_raw_len <= ARTIFACT_RESULT_MAX);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+
+ /* Exact-cap bytes survive an injected EAGAIN. Once they flush, the next
+ * producer byte records the crossing and the retained 16 MiB stays exact. */
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p1) == 0);
+ nonblock_fd(p1[0]);
+ managed_run("shell_bg", "{\"command\":\"perl -e "
+ "'print q(x) x 17825792'\"}",
+ artifact_ids[6], p1[1], 1);
+ job_id = ctx->artifact_job_id;
+ job_test_backpressure_at_boundary();
+ spooled = 0;
+ clock_gettime(CLOCK_MONOTONIC, &start);
+ for (;;) {
+ job_pump(ctx->jobs);
+ spooled += drain_bytes(p1[0]);
+ if (job_test_boundary_pending(ctx->jobs, job_id))
+ break;
+ clock_gettime(CLOCK_MONOTONIC, &end);
+ if (elapsed_ms(&start, &end) >= 10000)
+ break;
+ if (job_test_last_pump_bytes(ctx->jobs, job_id) == 0)
+ usleep(1000);
+ }
+ if (!job_test_boundary_pending(ctx->jobs, job_id) &&
+ job_test_state(ctx->jobs, job_id, &cap_state) == 0)
+ fprintf(stderr, "cap witness: observed=%llu retained=%llu "
+ "pending=%zu cap=%d exited=%d producer=%d artifact=%d "
+ "storage_error=%d unavailable=%d\n",
+ (unsigned long long)cap_state.observed,
+ (unsigned long long)cap_state.retained,
+ cap_state.pending_bytes, cap_state.cap_reached,
+ cap_state.exited, cap_state.producer_open,
+ cap_state.artifact_open, cap_state.storage_error,
+ cap_state.storage_unavailable);
+ CHECK(job_test_boundary_pending(ctx->jobs, job_id));
+ for (i = 0; i < 10; i++) {
+ spooled += drain_bytes(p1[0]);
+ job_pump(ctx->jobs);
+ }
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ managed_run("shell_output", arg, artifact_ids[7], -1, 0);
+ spooled += drain_bytes(p1[0]);
+ CHECK(err_flag && ctx->artifact_cap_reached &&
+ ctx->artifact_termination == ARTIFACT_TERM_CAP &&
+ ctx->artifact_observed == ARTIFACT_RAW_MAX + 1ULL &&
+ ctx->artifact_raw_len == ARTIFACT_RAW_MAX &&
+ spooled == ARTIFACT_RAW_MAX);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[6]) == 0);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+ close(p1[0]);
+
+ /* Backpressure has no arbitrary settlement deadline. Cancelling a poll
+ * leaves a committed job and its old spool intact, so a later Turn can
+ * drain and freeze the same bytes without loss. */
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p1) == 0);
+ nonblock_fd(p1[0]);
+ duplicate = dup(p1[1]);
+ CHECK(duplicate != -1);
+ managed_run("shell_bg", "{\"command\":\"yes blocked\"}",
+ artifact_ids[0], p1[1], 1);
+ job_id = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ fill_pipe(duplicate);
+ close(duplicate);
+ for (i = 0; i < 200 &&
+ job_test_pending_bytes(ctx->jobs, job_id) == 0; i++) {
+ job_pump(ctx->jobs);
+ usleep(5000);
+ }
+ CHECK(job_test_pending_bytes(ctx->jobs, job_id) > 0);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", job_id);
+ notifier = fork();
+ if (notifier == -1)
+ err(1, "fork background cancel notifier");
+ if (notifier == 0) {
+ usleep(150000);
+ (void)kill(getppid(), SIGUSR1);
+ _exit(0);
+ }
+ clock_gettime(CLOCK_MONOTONIC, &start);
+ managed_run("shell_output", arg, artifact_ids[1], nullfd, 1);
+ clock_gettime(CLOCK_MONOTONIC, &end);
+ (void)waitpid(notifier, &status, 0);
+ CHECK(!err_flag && !ctx->artifact_captured &&
+ !ctx->artifact_spool_adopted && elapsed_ms(&start, &end) < 1500);
+ CHECK(job_test_state(ctx->jobs, job_id, &cap_state) == 0 &&
+ !cap_state.storage_error && !cap_state.exited &&
+ cap_state.producer_open && cap_state.artifact_open &&
+ cap_state.pending_bytes > 0);
+ frozen_expected = cap_state.observed;
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+ notifier = fork();
+ if (notifier == -1)
+ err(1, "fork Artifact drain helper");
+ if (notifier == 0) {
+ ssize_t n;
+
+ close(p1[1]);
+ for (;;) {
+ n = read(p1[0], sink, sizeof(sink));
+ if (n > 0 || (n == -1 && errno == EINTR))
+ continue;
+ if (n == -1 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
+ usleep(1000);
+ continue;
+ }
+ _exit(n == 0 ? 0 : 2);
+ }
+ }
+ close(p1[0]);
+ snapshot = job_test_output_bytes(ctx->jobs, job_id);
+ CHECK(snapshot > 0 && snapshot <= job_test_pump_per_job_limit());
+ frozen_expected += snapshot;
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[1], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_captured &&
+ ctx->artifact_spool_adopted && ctx->artifact_spool_detach &&
+ ctx->artifact_observed == frozen_expected &&
+ ctx->artifact_result_available &&
+ strcmp(ctx->artifact_result_id, artifact_ids[0]) == 0);
+ (void)waitpid(notifier, &status, 0);
+ CHECK(WIFEXITED(status) && WEXITSTATUS(status) == 0);
+ CHECK(job_artifact_release(ctx->jobs, job_id, artifact_ids[0]) == 0);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+ run("shell_kill", arg);
+ CHECK(!err_flag);
+
+ /* Foreground writes share the execution deadline and SIGUSR1 cancels only
+ * the active group; the persistent executor has no poisoned next-call latch. */
+ CHECK(pipe(p1) == 0);
+ fill_pipe(p1[1]);
+ ctx->shell_timeout_ms = 300;
+ clock_gettime(CLOCK_MONOTONIC, &start);
+ managed_run("shell", "{\"command\":\"printf x; sleep 30\"}",
+ artifact_ids[2], p1[1], 1);
+ clock_gettime(CLOCK_MONOTONIC, &end);
+ CHECK(err_flag && ctx->artifact_termination == ARTIFACT_TERM_TIMEOUT &&
+ elapsed_ms(&start, &end) < 1500);
+ managed_clear();
+ close(p1[0]);
+
+ CHECK(pipe(p1) == 0);
+ fill_pipe(p1[1]);
+ ctx->shell_timeout_ms = 5000;
+ notifier = fork();
+ if (notifier == -1)
+ err(1, "fork cancel notifier");
+ if (notifier == 0) {
+ usleep(150000);
+ (void)kill(getppid(), SIGUSR1);
+ _exit(0);
+ }
+ clock_gettime(CLOCK_MONOTONIC, &start);
+ managed_run("shell", "{\"command\":\"printf x; sleep 30\"}",
+ artifact_ids[3], p1[1], 1);
+ clock_gettime(CLOCK_MONOTONIC, &end);
+ (void)waitpid(notifier, &status, 0);
+ CHECK(elapsed_ms(&start, &end) < 1500);
+ managed_clear();
+ close(p1[0]);
+ ctx->shell_timeout_ms = 0;
+ run("shell", "{\"command\":\"printf next-tool-ok\"}");
+ CHECK(!err_flag && outhas("next-tool-ok"));
+}
+
+static void
+test_bounded_job_retirement(void)
+{
+ struct tool_ctx *saved = ctx, *bounded;
+ struct artifact_spool_drop drop, bad_drop;
+ struct job_test_state state;
+ char arg[64], zero_id[FUGU_ARTIFACT_ID_BUFSZ] = { 0 };
+ int p[2], old_job, i, nullfd, settled;
+ uint32_t old_correlation, correlation;
+ uint64_t spooled;
+ ssize_t n;
+
+ bounded = tool_ctx_new(1, 0);
+ ctx = bounded;
+ job_set_cap(ctx->jobs, 4);
+ job_test_set_record_limit(ctx->jobs, 1);
+ job_test_set_tombstone_limit(ctx->jobs, 4);
+
+ /* The record bound is independent of the running-job cap. Admitting a
+ * second job retires the completed, unpolled first job and publishes its
+ * exact current-spool key exactly once. */
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p) == 0);
+ nonblock_fd(p[0]);
+ managed_run("shell_bg", "{\"command\":\"printf retired-by-bound\"}",
+ artifact_ids[0], p[1], 1);
+ old_job = ctx->artifact_job_id;
+ old_correlation = ctx->artifact_correlation;
+ CHECK(!err_flag && old_job > 0 && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ spooled = 0;
+ settled = 0;
+ for (i = 0; i < 200; i++) {
+ job_pump(ctx->jobs);
+ spooled += drain_bytes(p[0]);
+ if (job_test_state(ctx->jobs, old_job, &state) == 0 &&
+ state.exited && !state.producer_open && !state.artifact_open &&
+ state.pending_bytes == 0) {
+ settled = 1;
+ break;
+ }
+ usleep(5000);
+ }
+ CHECK(settled && spooled == strlen("retired-by-bound"));
+ n = read(p[0], &i, 1);
+ CHECK(n == 0);
+ close(p[0]);
+
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[1], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_job_id != old_job &&
+ job_test_record_count(ctx->jobs) == 1 &&
+ job_test_tombstone_count(ctx->jobs) == 1);
+ memset(&drop, 0, sizeof(drop));
+ correlation = 0;
+ CHECK(job_artifact_drop_next(ctx->jobs, &correlation, &drop) == 1 &&
+ correlation == old_correlation && drop.job_id == (uint32_t)old_job &&
+ strcmp(drop.artifact_id, artifact_ids[0]) == 0);
+ CHECK(job_artifact_drop_next(ctx->jobs, &correlation, &drop) == 0);
+ managed_clear();
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", old_job);
+ run("shell_output", arg);
+ CHECK(err_flag && outhas("no such job"));
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[0]) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[2]) == -1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[2]) == -1);
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation + 1, &drop) == -1);
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &drop) == 0 &&
+ job_test_tombstone_count(ctx->jobs) == 0);
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &drop) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[0]) == -1);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* STORE_ERROR races retire the same kind of current DROP tombstone with
+ * one exact CANCEL instead of DROP_ACK. When the DROP is still unsent,
+ * cancellation removes it before main emits the exact CANCEL_ACK. The two
+ * terminal controls are mutually exclusive, and RELEASE is never valid for
+ * an unpolled current spool. */
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p) == 0);
+ nonblock_fd(p[0]);
+ managed_run("shell_bg", "{\"command\":\"printf cancelled-drop\"}",
+ artifact_ids[5], p[1], 1);
+ old_job = ctx->artifact_job_id;
+ old_correlation = ctx->artifact_correlation;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ settled = 0;
+ for (i = 0; i < 200; i++) {
+ job_pump(ctx->jobs);
+ (void)drain_bytes(p[0]);
+ if (job_test_state(ctx->jobs, old_job, &state) == 0 &&
+ state.exited && !state.producer_open && !state.artifact_open &&
+ state.pending_bytes == 0) {
+ settled = 1;
+ break;
+ }
+ usleep(5000);
+ }
+ CHECK(settled);
+ close(p[0]);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[6], nullfd, 1);
+ CHECK(!err_flag && job_test_record_count(ctx->jobs) == 1);
+ memset(&drop, 0, sizeof(drop));
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[5]) == 0 &&
+ job_test_tombstone_count(ctx->jobs) == 0);
+ correlation = 0;
+ CHECK(job_artifact_drop_next(ctx->jobs, &correlation, &drop) == 0);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[5]) == -1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[5]) == -1);
+ drop.job_id = (uint32_t)old_job;
+ strlcpy(drop.artifact_id, artifact_ids[5], sizeof(drop.artifact_id));
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &drop) == -1);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* Live current keys, prior-poll history, and tombstones share one global
+ * ownership bound. A repeated running poll at the bound stops without
+ * adopting another key and retains the current key as one settlement slot. */
+ job_test_set_ownership_limit(ctx->jobs, 2);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"printf bounded; sleep 30\"}",
+ artifact_ids[0], nullfd, 1);
+ old_job = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 1);
+ managed_clear();
+ for (i = 0; i < 20; i++) {
+ job_pump(ctx->jobs);
+ usleep(5000);
+ }
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", old_job);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[1], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_spool_adopted &&
+ ctx->artifact_result_available &&
+ strcmp(ctx->artifact_result_id, artifact_ids[0]) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 2);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[2], nullfd, 1);
+ CHECK(err_flag && ctx->artifact_captured &&
+ ctx->artifact_termination == ARTIFACT_TERM_CANCELLED &&
+ !ctx->artifact_spool_adopted && ctx->artifact_result_available &&
+ strcmp(ctx->artifact_result_id, artifact_ids[1]) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 2 &&
+ job_test_record_count(ctx->jobs) == 1);
+ managed_clear();
+ run("shell_output", arg);
+ CHECK(err_flag && outhas("awaiting Artifact settlement") &&
+ job_test_ownership_count(ctx->jobs) == 2);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[0]) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[0]) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[0]) == -1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[1]) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 0 &&
+ job_test_record_count(ctx->jobs) == 0);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[1]) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[1]) == -1);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* The same no-replacement result identifies an unavailable old spool by
+ * job plus the all-zero id. Tool has no exact available key to settle;
+ * the Coordinator can retire its unavailable reservation from this result. */
+ job_test_set_ownership_limit(ctx->jobs, 1);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[2], -1, 0);
+ old_job = ctx->artifact_job_id;
+ CHECK(!err_flag && old_job > 0 && job_turn_commit(ctx->jobs) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 1);
+ managed_clear();
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", old_job);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[3], nullfd, 1);
+ CHECK(err_flag && ctx->artifact_captured &&
+ ctx->artifact_termination == ARTIFACT_TERM_CANCELLED &&
+ ctx->artifact_job_id == old_job && !ctx->artifact_spool_adopted &&
+ !ctx->artifact_result_available && ctx->artifact_result_id[0] == '\0' &&
+ job_test_ownership_count(ctx->jobs) == 0 &&
+ job_test_record_count(ctx->jobs) == 0);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* One valid STORE_ERROR cancellation is job-wide. Matching an old live
+ * history key retires every other history and the current replacement; no
+ * second control for that job may succeed. */
+ job_test_set_ownership_limit(ctx->jobs, 16);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[0], nullfd, 1);
+ old_job = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", old_job);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[1], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_spool_adopted);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[2], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_spool_adopted &&
+ job_test_ownership_count(ctx->jobs) == 3);
+ managed_clear();
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[0]) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 0);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[0]) == -1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[1]) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[2]) == -1);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[3], nullfd, 1);
+ CHECK(err_flag && ctx->artifact_termination == ARTIFACT_TERM_STORAGE_ERROR &&
+ job_test_record_count(ctx->jobs) == 0);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* The same job-wide rule covers an archived job with both historical
+ * tombstones and a published current DROP. Socket ordering puts that DROP
+ * before main's exact CANCEL_ACK; cancellation then makes DROP_ACK and every
+ * sibling control stale. */
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"printf archived; sleep 1\"}",
+ artifact_ids[0], nullfd, 1);
+ old_job = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", old_job);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_output", arg, artifact_ids[1], nullfd, 1);
+ old_correlation = ctx->artifact_correlation;
+ CHECK(!err_flag && ctx->artifact_spool_adopted &&
+ job_test_ownership_count(ctx->jobs) == 2);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ settled = 0;
+ for (i = 0; i < 300; i++) {
+ job_pump(ctx->jobs);
+ if (job_test_state(ctx->jobs, old_job, &state) == 0 &&
+ state.exited && !state.producer_open && !state.artifact_open &&
+ state.pending_bytes == 0) {
+ settled = 1;
+ break;
+ }
+ usleep(5000);
+ }
+ CHECK(settled);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[4], nullfd, 1);
+ CHECK(!err_flag && job_test_record_count(ctx->jobs) == 1 &&
+ job_test_tombstone_count(ctx->jobs) == 2);
+ memset(&drop, 0, sizeof(drop));
+ correlation = 0;
+ CHECK(job_artifact_drop_next(ctx->jobs, &correlation, &drop) == 1 &&
+ correlation == old_correlation && drop.job_id == (uint32_t)old_job &&
+ strcmp(drop.artifact_id, artifact_ids[1]) == 0);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[0]) == 0 &&
+ job_test_tombstone_count(ctx->jobs) == 0);
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &drop) == -1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[0]) == -1);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, artifact_ids[1]) == -1);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 0);
+
+ /* Unavailable reservations are retained ownership too. Their retirement
+ * uses the same exact correlation/job DROP, with only an all-zero id valid. */
+ job_test_set_ownership_limit(ctx->jobs, 16);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ managed_run("shell_bg", "{\"command\":\"printf unavailable-drop\"}",
+ artifact_ids[3], -1, 0);
+ old_job = ctx->artifact_job_id;
+ old_correlation = ctx->artifact_correlation;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 1);
+ managed_clear();
+ settled = 0;
+ for (i = 0; i < 200; i++) {
+ job_pump(ctx->jobs);
+ if (job_test_state(ctx->jobs, old_job, &state) == 0 &&
+ state.exited && !state.producer_open && !state.artifact_open &&
+ state.pending_bytes == 0) {
+ settled = 1;
+ break;
+ }
+ usleep(5000);
+ }
+ CHECK(settled);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[4], nullfd, 1);
+ CHECK(!err_flag && job_test_record_count(ctx->jobs) == 1 &&
+ job_test_tombstone_count(ctx->jobs) == 1);
+ memset(&drop, 0, sizeof(drop));
+ correlation = 0;
+ CHECK(job_artifact_drop_next(ctx->jobs, &correlation, &drop) == 1 &&
+ correlation == old_correlation && drop.job_id == (uint32_t)old_job &&
+ memcmp(drop.artifact_id, zero_id, sizeof(zero_id)) == 0);
+ CHECK(job_artifact_cancel(ctx->jobs, old_job, drop.artifact_id) == -1);
+ bad_drop = drop;
+ bad_drop.artifact_id[0] = 'a';
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &bad_drop) == -1);
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &drop) == 0 &&
+ job_test_tombstone_count(ctx->jobs) == 0);
+ CHECK(job_artifact_drop_ack(ctx->jobs, old_correlation, &drop) == -1);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0 &&
+ job_test_ownership_count(ctx->jobs) == 0);
+
+ /* A full exact-key tombstone table cannot grow without bound. With no
+ * running jobs, admission fails as settlement backlog rather than
+ * pretending the independent max_bg_jobs concurrency gate was reached. */
+ job_test_set_tombstone_limit(ctx->jobs, 0);
+ CHECK(job_turn_begin(ctx->jobs) == 0 && pipe(p) == 0);
+ nonblock_fd(p[0]);
+ managed_run("shell_bg", "{\"command\":\"printf held-by-backlog\"}",
+ artifact_ids[2], p[1], 1);
+ old_job = ctx->artifact_job_id;
+ CHECK(!err_flag && job_turn_commit(ctx->jobs) == 0);
+ managed_clear();
+ settled = 0;
+ for (i = 0; i < 200; i++) {
+ job_pump(ctx->jobs);
+ (void)drain_bytes(p[0]);
+ if (job_test_state(ctx->jobs, old_job, &state) == 0 &&
+ state.exited && !state.producer_open && !state.artifact_open &&
+ state.pending_bytes == 0) {
+ settled = 1;
+ break;
+ }
+ usleep(5000);
+ }
+ CHECK(settled);
+ close(p[0]);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ managed_run("shell_bg", "{\"command\":\"sleep 30\"}",
+ artifact_ids[3], nullfd, 1);
+ CHECK(err_flag && outhas("settlement backlog") &&
+ !outhas("background-job cap") &&
+ job_test_record_count(ctx->jobs) == 1 &&
+ job_test_tombstone_count(ctx->jobs) == 0);
+ managed_clear();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+
+ /* A final poll can still settle that retained live record; once its exact
+ * control arrives, no tombstone is needed and the record disappears. */
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ nullfd = open("/dev/null", O_WRONLY);
+ CHECK(nullfd != -1);
+ (void)snprintf(arg, sizeof(arg), "{\"id\":%d}", old_job);
+ managed_run("shell_output", arg, artifact_ids[4], nullfd, 1);
+ CHECK(!err_flag && ctx->artifact_termination == ARTIFACT_TERM_EXIT &&
+ job_test_record_count(ctx->jobs) == 1);
+ CHECK(job_artifact_release(ctx->jobs, old_job, artifact_ids[2]) == 0 &&
+ job_test_record_count(ctx->jobs) == 0);
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ managed_clear();
+
+ tool_ctx_free(bounded);
+ ctx = saved;
+}
+
+static void
+test_background_pump_fairness(void)
+{
+ struct tool_ctx *saved = ctx, *fair;
+ int i, round, seen[3] = { 0, 0, 0 };
+
+ fair = tool_ctx_new(1, 0);
+ ctx = fair;
+ job_set_cap(ctx->jobs, 3);
+ CHECK(job_test_pump_per_job_limit() == 64 * 1024);
+ CHECK(job_test_pump_total_limit() == 1024 * 1024);
+ CHECK(job_turn_begin(ctx->jobs) == 0);
+ for (i = 0; i < 3; i++) {
+ run("shell_bg", "{\"command\":\"yes fair\"}");
+ CHECK(!err_flag && ctx->artifact_job_id == i + 1);
+ }
+ for (round = 0; round < 200; round++) {
+ for (i = 0; i < 3; i++)
+ if (job_test_output_bytes(ctx->jobs, i + 1) < 4096)
+ break;
+ if (i == 3)
+ break;
+ usleep(5000);
+ }
+ CHECK(i == 3);
+ /* Compress the same scheduler policy so two jobs saturate one pump;
+ * the next pump must begin with the skipped third job. */
+ job_test_set_pump_limits(4096, 8192);
+ for (round = 0; round < 2; round++) {
+ job_pump(ctx->jobs);
+ CHECK(job_test_last_pump_total(ctx->jobs) == 8192);
+ for (i = 0; i < 3; i++) {
+ size_t n = job_test_last_pump_bytes(ctx->jobs, i + 1);
+
+ CHECK(n <= 4096);
+ if (n > 0)
+ seen[i] = 1;
+ }
+ if (round == 0)
+ CHECK(job_test_last_pump_bytes(ctx->jobs, 1) == 4096 &&
+ job_test_last_pump_bytes(ctx->jobs, 2) == 4096 &&
+ job_test_last_pump_bytes(ctx->jobs, 3) == 0);
+ else
+ CHECK(job_test_last_pump_bytes(ctx->jobs, 3) == 4096);
+ }
+ CHECK(seen[0] && seen[1] && seen[2]);
+ job_test_reset_pump_limits();
+ CHECK(job_turn_abort(ctx->jobs) == 0);
+ tool_ctx_free(fair);
+ ctx = saved;
+}
+
int
main(int argc, char *argv[])
{
+ struct sigaction sa;
+
log_init(LOG_TO_STDERR, 0, LOG_USER);
if (argc != 2)
errx(1, "usage: tools_test <tree>");
if (chdir(argv[1]) == -1)
err(1, "chdir %s", argv[1]);
+ memset(&sa, 0, sizeof(sa));
+ sa.sa_handler = cancel_active_signal;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(SIGUSR1, &sa, NULL) == -1)
+ err(1, "sigaction SIGUSR1");
buf_init(&out);
CHECK(err_flag && outhas("no such job")); /* abort removed it */
run("shell_kill", "{\"id\":999}");
CHECK(err_flag && outhas("no such job"));
+ test_background_pump_fairness();
+ test_managed_artifacts();
+ test_bounded_job_retirement();
/* --- unknown tool ----------------------------------------- */
run("frobnicate", "{}");
blob - 262a0930c5ca8ea077a200efe4dcabb185a30b21
blob + 7ec62a2cb9334534b80aa371ac9fb0dd5858aae6
--- regress/turn/reqcheck.c
+++ regress/turn/reqcheck.c
return (ok);
}
+static int
+artifact_text_is(const struct json *j, int tok, const char *want)
+{
+ struct json nested;
+ char *serialized;
+ size_t len;
+ int root, ok = 0;
+
+ if (tok == -1 || !json_is_string(j, tok) ||
+ (serialized = json_get_str(j, tok, &len)) == NULL)
+ return (0);
+ if (json_parse(&nested, serialized, len, 0) != 0)
+ goto done;
+ root = json_root(&nested);
+ ok = json_is_object(&nested, root) &&
+ string_is(&nested, json_obj_get(&nested, root, "preview_head"),
+ want) &&
+ string_is(&nested, json_obj_get(&nested, root, "preview_tail"), "") &&
+ string_is(&nested, json_obj_get(&nested, root, "preview_encoding"),
+ "utf-8") &&
+ json_get_bool(&nested,
+ json_obj_get(&nested, root, "preview_abbreviated")) == 0;
+ json_done(&nested);
+done:
+ free(serialized);
+ return (ok);
+}
+
static const u_char *
request_body(const struct buf *b, size_t *len)
{
!json_is_object(j, block) ||
!string_is(j, json_obj_get(j, block, "type"), "tool_result") ||
!string_is(j, json_obj_get(j, block, "tool_use_id"), tool_id) ||
- !string_is(j, json_obj_get(j, block, "content"), "slept-ok\n") ||
+ !artifact_text_is(j, json_obj_get(j, block, "content"),
+ "slept-ok\n") ||
(block = json_arr_get(j, content, 1)) == -1 ||
!json_is_object(j, block) ||
!string_is(j, json_obj_get(j, block, "type"), "text") ||
!json_is_object(j, block) ||
!string_is(j, json_obj_get(j, block, "type"), "tool_result") ||
!string_is(j, json_obj_get(j, block, "tool_use_id"), "toolu_zz") ||
- !string_is(j, json_obj_get(j, block, "content"), "slept-ok\n"))
+ !artifact_text_is(j, json_obj_get(j, block, "content"),
+ "slept-ok\n"))
return (0);
return (1);
}
blob - a59368d8a61b93cd383478cf92563bee950aa163
blob + 47b00ed92f85eeb33c03f3fb4ad6a84b91973756
--- regress/turn/run.sh
+++ regress/turn/run.sh
ok() { if [ "$2" -eq 0 ]; then pass "$1"; else fail "$1"; fi; }
dir=$(mktemp -d /tmp/fugu-turn.XXXXXXXXXX)
+test_home=$(mktemp -d /tmp/fugu-turn-home.XXXXXXXXXX)
stubpid=''
+fugupid=''
cleanup() {
+ [ -n "$fugupid" ] && { kill "$fugupid" 2>/dev/null; wait "$fugupid" 2>/dev/null; }
[ -n "$stubpid" ] && { kill "$stubpid" 2>/dev/null; wait "$stubpid" 2>/dev/null; }
- rm -rf "$dir"
+ rm -rf "$dir" "$test_home"
}
trap cleanup EXIT INT TERM
# stage the real binaries
lx=$dir/libexec/fugu
mkdir -p "$lx"
-for w in fugu-tty fugu-api fugu-web fugu-tool fugu-editor; do
+for w in fugu-tty fugu-api fugu-web fugu-tool fugu-artifact fugu-editor; do
b=$root/src/$w/obj/$w
[ -x "$b" ] || b=$root/src/$w/$w
[ -x "$b" ] || { echo "FAIL - $w not built"; exit 1; }
fugu=$root/src/fugu/obj/fugu
[ -x "$fugu" ] || fugu=$root/src/fugu/fugu
-export HOME=$dir/home
-mkdir -p "$HOME"
+export HOME=$test_home
KEY=sk-test-abc123XYZ
start_stub() { # $1 = scenario words
cp "$dir/fugu-editor.real" "$lx/fugu-editor"
stop_stub
+# The Lead Tool's project and fixed runtime unveils must never cover managed
+# Artifact storage. Refuse each physical overlap before spawning a worker,
+# opening a Journal, or contacting the Provider.
+owner_home=$HOME
+rm -rf "$owner_home/.fugu"
+rm -f "$dir/reqout"
+start_stub "OK"
+write_conf "$stubport"
+( cd "$owner_home" && FUGU_CONF=$dir/fugu.conf \
+ FUGU_LIBEXEC=$dir/libexec/fugu FUGU_CA_FILE=$dir/cert.pem \
+ "$fugu" -p "scope probe" </dev/null >"$dir/out" 2>"$dir/errout" )
+rc=$?
+ok "Artifact scope: HOME as project root is refused" \
+ $([ "$rc" -ne 0 ]; echo $?)
+grep -q "Artifact storage overlaps Tool filesystem authority" "$dir/errout"
+ok "Artifact scope: overlap refusal is conspicuous" $?
+! test -e "$dir/reqout" &&
+ ! ls "$owner_home/.fugu/sessions"/*.ndjson >/dev/null 2>&1
+ok "Artifact scope: refusal precedes Journal and Provider effects" $?
+stop_stub
+
+rm -rf "$owner_home/.fugu"
+mkdir -p "$owner_home/.fugu/artifacts/project"
+printf 'private-marker\n' > "$owner_home/.fugu/artifacts/raw-marker"
+rm -f "$dir/reqout"
+start_stub "OK"
+write_conf "$stubport"
+( cd "$owner_home/.fugu/artifacts/project" && \
+ FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem "$fugu" -p "inside probe" \
+ </dev/null >"$dir/out" 2>"$dir/errout" )
+rc=$?
+ok "Artifact scope: project inside the managed root is refused" \
+ $([ "$rc" -ne 0 ]; echo $?)
+test "$(cat "$owner_home/.fugu/artifacts/raw-marker")" = private-marker &&
+ ! test -e "$dir/reqout"
+ok "Artifact scope: refusal leaves raw storage untouched" $?
+stop_stub
+
+rm -rf "$owner_home/.fugu" "$dir/home-alias"
+ln -s "$owner_home" "$dir/home-alias"
+rm -f "$dir/reqout"
+start_stub "OK"
+write_conf "$stubport"
+( cd "$owner_home" && HOME="$dir/home-alias" \
+ FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem "$fugu" -p "alias probe" \
+ </dev/null >"$dir/out" 2>"$dir/errout" )
+rc=$?
+ok "Artifact scope: HOME symlink aliases cannot hide an overlap" \
+ $([ "$rc" -ne 0 ]; echo $?)
+! test -e "$dir/reqout" &&
+ ! ls "$owner_home/.fugu/sessions"/*.ndjson >/dev/null 2>&1
+ok "Artifact scope: aliased refusal has no external side effect" $?
+stop_stub
+
+rm -f "$dir/reqout"
+start_stub "OK"
+write_conf "$stubport"
+HOME=/usr FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem "$fugu" -p "fixed root probe" \
+ </dev/null >"$dir/out" 2>"$dir/errout"
+rc=$?
+ok "Artifact scope: HOME below a fixed Tool unveil is refused" \
+ $([ "$rc" -ne 0 ]; echo $?)
+grep -q "Artifact storage overlaps Tool filesystem authority" "$dir/errout" &&
+ ! test -e "$dir/reqout"
+ok "Artifact scope: fixed-root refusal precedes Provider access" $?
+stop_stub
+
# --- 1: a clean OK turn ---------------------------------------------
start_stub "OK"
write_conf "$stubport"
$([ "$n" -eq 2 ]; echo $?)
stop_stub
+# --- 13-artifact: abbreviated raw output stays path-free -------------
+rm -rf "$HOME/.fugu"
+start_stub "ARTIFACTTOOL OUTPUTREAD OUTPUTREADDEFAULT OK"
+write_conf "$stubport"
+run_fugu -p "produce retained output"
+ok "artifact tool loop: abbreviated result completes" \
+ $([ "$rc" -eq 0 ]; echo $?)
+request_log_entry 2 "$dir/artifact-request"
+grep -Fq '\"preview_abbreviated\":true' "$dir/artifact-request"
+ok "artifact tool loop: next Generation receives an abbreviated result" $?
+grep -Eq '\\"id\\":\\"[0-9a-f]{32}\\"' "$dir/artifact-request"
+ok "artifact tool loop: result exposes only an opaque output id" $?
+journal=$(ls "$sessions"/*.ndjson 2>/dev/null | head -1)
+grep -a -E '(\.fugu/artifacts|\.pending-)' \
+ "$dir/artifact-request" "$journal" "$dir/out" "$dir/errout" \
+ >/dev/null 2>&1
+ok "artifact tool loop: no private storage path crosses runtime surfaces" \
+ $([ $? -ne 0 ]; echo $?)
+grep -q '"t":"artifact"' "$journal"
+ok "artifact tool loop: the structured Artifact record commits" $?
+request_log_entry 3 "$dir/output-read-request"
+grep -Fq '"name":"output_read"' "$dir/output-read-request" &&
+ grep -Fq '"offset":65536' "$dir/output-read-request" &&
+ grep -Fq '"limit":4' "$dir/output-read-request" &&
+ grep -Fq '"encoding":"base64"' "$dir/output-read-request"
+ok "output_read loop: Provider arguments cross the Coordinator seam" $?
+grep -Fq '\"offset\":65536' "$dir/output-read-request" &&
+ grep -Fq '\"returned_bytes\":4' "$dir/output-read-request" &&
+ grep -Fq '\"next_offset\":65540' "$dir/output-read-request" &&
+ grep -Fq '\"state\":\"pending\"' "$dir/output-read-request" &&
+ grep -Fq '\"data\":\"QQD/Qg==\"' \
+ "$dir/output-read-request"
+ok "output_read loop: pending base64 page reaches the next Generation" $?
+request_log_entry 4 "$dir/output-read-default-request"
+grep -Eq '"id":"toolu_output_read_default","name":"output_read",'\
+'"input":\{"id":"[0-9a-f]{32}"\}' "$dir/output-read-default-request" &&
+ grep -Fq '\"offset\":0' "$dir/output-read-default-request" &&
+ grep -Fq '\"returned_bytes\":65536' \
+ "$dir/output-read-default-request" &&
+ grep -Fq '\"next_offset\":65536' \
+ "$dir/output-read-default-request" &&
+ grep -Fq '\"encoding\":\"text\"' \
+ "$dir/output-read-default-request" &&
+ grep -Fq '\"state\":\"pending\"' \
+ "$dir/output-read-default-request"
+ok "output_read loop: id-only call applies text and 64 KiB defaults" $?
+stop_stub
+
+# A read response remains an exact bounded transaction after Ctrl-C. The
+# regression-only Role delay widens the cancellation window; Coordinator must
+# drain DATA/READ_RESULT before Artifact Turn abort reuses the same channel.
+rm -rf "$HOME/.fugu"
+start_stub "ARTIFACTTOOL OUTPUTREAD OK"
+write_conf "$stubport"
+FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \
+ FUGU_ARTIFACT_READ_DELAY_MS=1000 \
+ "$fugu" -p "cancel retained output read" </dev/null \
+ >"$dir/out" 2>"$dir/errout" &
+fugupid=$!
+i=0
+while ! grep -q '^--- OUTPUTREAD ---$' "$dir/reqlog" 2>/dev/null &&
+ [ "$i" -lt 100 ]; do
+ sleep 0.05
+ i=$((i + 1))
+done
+sleep 0.1
+/bin/kill -INT "$fugupid" 2>/dev/null
+wait "$fugupid"
+rc=$?
+fugupid=''
+ok "output_read cancel: interrupted read exits 3" \
+ $([ "$rc" -eq 3 ]; echo $?)
+if grep -Eq 'bad Artifact read|unexpected Artifact event' \
+ "$dir/errout" 2>/dev/null; then
+ fail "output_read cancel: exact delayed response drains before rollback"
+else
+ pass "output_read cancel: exact delayed response drains before rollback"
+fi
+stop_stub
+
# --- 13a: a full brokered web loop over the real mesh (M8) ----------
# The development-only private-address seam lets the isolated fixture act
# as both provider and web origin; installed/setgid binaries ignore it.
# --- 23b: automatic Compaction preflight ---------------------------
# With the minimal five-tool catalogue below, the conservative estimate for
-# System "S" plus User "strict boundary" is exactly 463 tokens. Equality at
+# System "S" plus User "strict boundary" is exactly 591 tokens. Equality at
# window-reserve is intentionally not a cut.
rm -rf "$HOME/.fugu"
start_stub "OK"
web_search no
max_bg_jobs 0
max_subagents 0
-context_limit 464
+context_limit 592
compact_reserve_tokens 1
compact_keep_tokens 1
auto_compact yes
$([ "$n" -eq 1 ]; echo $?)
stop_stub
-# A 1,000-byte first User request is 709 estimated tokens, below the 750-token
-# threshold. Its completed pair plus the second User reaches 765; replacing
-# that history with short notes falls to 502 before the second Generation.
+# A 1,000-byte first User request is 837 estimated tokens, below the 878-token
+# threshold. Its completed pair plus the second User reaches 893; replacing
+# that history with short notes falls to 630 before the second Generation.
rm -rf "$HOME/.fugu"
start_stub "OK SUMMARY OK"
cat > "$dir/fugu.conf" <<EOF
web_search no
max_bg_jobs 0
max_subagents 0
-context_limit 751
+context_limit 879
compact_reserve_tokens 1
compact_keep_tokens 1
auto_compact yes
# A fresh projection is estimated again after Tool results. The 1,400-byte
# Owner prefix fits before the Tool call, the pair crosses the threshold, and
-# replacing that prefix with short notes falls below it again. The retained
-# Assistant/result pair is not re-executed; only the summary and continuation
-# Generation are additional Provider calls.
+# replacing that prefix with short notes falls below it again. The 150-token
+# suffix budget retains the generated Artifact result with its Assistant; the
+# pair is not re-executed, and only the summary and continuation Generation are
+# additional Provider calls.
rm -rf "$HOME/.fugu"
start_stub "TOOLUSE SUMMARY OK"
cat > "$dir/fugu.conf" <<EOF
web_search no
max_bg_jobs 0
max_subagents 0
-context_limit 851
+context_limit 979
compact_reserve_tokens 1
-compact_keep_tokens 100
+compact_keep_tokens 150
auto_compact yes
effort "high"
EOF
$([ "$n" = 1 ]; echo $?)
stop_stub
+# A committed job is polled in a later Turn. Its replacement spool is
+# detached by the out-of-band ROTATE frame before the ordinary Tool result;
+# settling the frozen segment must neither duplicate DETACH nor lose the job.
+rm -rf "$HOME/.fugu"
+rm -f "$dir/reqout"
+start_stub "BGTOOL OK JOBOUTPUT OK"
+write_conf "$stubport"
+printf 'launch committed background\nprobe committed background\n/quit\n' \
+ > "$dir/lin"
+run_fugu_in "$dir/lin"
+ok "background rotation: committed job poll completes" \
+ $([ "$rc" -eq 0 ]; echo $?)
+grep -q '"tool_use_id":"toolu_poll"' "$dir/reqout" 2>/dev/null
+ok "background rotation: frozen result reaches the provider" $?
+n=$(grep -c '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null)
+ok "background rotation: both Turns commit" \
+ $([ "$n" = 2 ]; echo $?)
+if grep -Eq 'bad Artifact|mismatched|duplicate' "$dir/errout" 2>/dev/null; then
+ fail "background rotation: ownership protocol stays singular"
+else
+ pass "background rotation: ownership protocol stays singular"
+fi
+stop_stub
+
+# A committed short job retires its current available spool while the Lead is
+# waiting through a Provider retry. DROP is asynchronous to Tool results;
+# Coordinator must discard only that exact current spool and ACK it once.
+rm -rf "$HOME/.fugu"
+rm -f "$dir/reqout"
+start_stub "BGSHORT OK RETRY429 OK"
+write_conf "$stubport"
+printf 'launch short background\nwait for available retirement\n/quit\n' \
+ > "$dir/lin"
+run_fugu_in "$dir/lin"
+ok "background retirement: available DROP round-trip completes" \
+ $([ "$rc" -eq 0 ]; echo $?)
+n=$(grep -c '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null)
+ok "background retirement: both available-DROP Turns commit" \
+ $([ "$n" = 2 ]; echo $?)
+if grep -Eq 'bad Artifact|mismatched|stale Artifact' \
+ "$dir/errout" 2>/dev/null; then
+ fail "background retirement: available key is acknowledged once"
+else
+ pass "background retirement: available key is acknowledged once"
+fi
+stop_stub
+
+# Eight detached logical reservations fill the 128 MiB Session quota without
+# writing 128 MiB of physical data. The ninth job receives an unavailable
+# zero-id reservation, commits, and later retires it over the same exact
+# DROP/ACK transaction without waiting for an Artifact terminal.
+rm -rf "$HOME/.fugu"
+rm -f "$dir/reqout"
+scenario=''
+: > "$dir/lin"
+i=0
+while [ "$i" -lt 8 ]; do
+ scenario="$scenario BGTOOL OK"
+ printf 'reserve background spool %s\n' "$i" >> "$dir/lin"
+ i=$((i + 1))
+done
+scenario="$scenario BGSHORT OK RETRY429 OK"
+printf 'launch unavailable short background\nwait for unavailable retirement\n/quit\n' \
+ >> "$dir/lin"
+start_stub "$scenario"
+write_conf "$stubport"
+printf 'max_bg_jobs 9\n' >> "$dir/fugu.conf"
+run_fugu_in "$dir/lin"
+ok "background retirement: unavailable DROP round-trip completes" \
+ $([ "$rc" -eq 0 ]; echo $?)
+n=$(grep -c '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null)
+ok "background retirement: all unavailable-DROP Turns commit" \
+ $([ "$n" = 10 ]; echo $?)
+if grep -Eq 'bad Artifact|mismatched|stale Artifact' \
+ "$dir/errout" 2>/dev/null; then
+ fail "background retirement: zero-id key is acknowledged once"
+else
+ pass "background retirement: zero-id key is acknowledged once"
+fi
+stop_stub
+
+# A ROTATE frame is ownership state in its own right. Publish it with a
+# blocking selftest send, then cancel before the ordinary Tool result. The
+# background job is still pending in this Turn, so rollback must consume the
+# frame without trying to release committed job ownership.
+rm -rf "$HOME/.fugu"
+rm -f "$dir/out" "$dir/errout" "$dir/reqout"
+start_stub "BGTOOL JOBOUTPUT"
+write_conf "$stubport"
+FUGU_SELFTEST=1 FUGU_ARTIFACT_FRAME0_DELAY_MS=3000 \
+ FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \
+ "$fugu" -d -p "cancel pending Artifact rotation" </dev/null \
+ >"$dir/out" 2>"$dir/errout" &
+fugupid=$!
+i=0
+while ! grep -Fq 'selftest frame0 Artifact rotation published' \
+ "$dir/errout" 2>/dev/null && [ "$i" -lt 200 ] && \
+ kill -0 "$fugupid" 2>/dev/null; do
+ sleep 0.05
+ i=$((i + 1))
+done
+grep -Fq 'selftest frame0 Artifact rotation published' \
+ "$dir/errout" 2>/dev/null
+reached=$?
+if [ "$reached" -eq 0 ]; then
+ /bin/kill -INT "$fugupid" 2>/dev/null
+else
+ /bin/kill -TERM "$fugupid" 2>/dev/null
+fi
+wait "$fugupid"
+rc=$?
+fugupid=''
+ok "background rotation cancel: pending frame publishes before Ctrl-C" \
+ "$reached"
+ok "background rotation cancel: pending Turn exits 3" \
+ $([ "$rc" -eq 3 ]; echo $?)
+n=$(grep -h -c '"outcome":"abandoned"' \
+ "$sessions"/*.ndjson 2>/dev/null | awk '{ n += $1 } END { print n + 0 }')
+ok "background rotation cancel: pending Turn is abandoned once" \
+ $([ "$n" -eq 1 ]; echo $?)
+n=$(grep -h -c '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null | \
+ awk '{ n += $1 } END { print n + 0 }')
+ok "background rotation cancel: pending job never commits" \
+ $([ "$n" -eq 0 ]; echo $?)
+n=$(grep -Fc 'selftest frame0 Artifact rotation published' \
+ "$dir/errout" 2>/dev/null)
+ok "background rotation cancel: pending frame publishes exactly once" \
+ $([ "$n" -eq 1 ]; echo $?)
+if grep -Eq 'selftest Artifact RELEASE|bad Artifact|mismatched Artifact|stale Artifact|missing cancelled Artifact' \
+ "$dir/errout" 2>/dev/null; then
+ fail "background rotation cancel: pending ownership rolls back cleanly"
+else
+ pass "background rotation cancel: pending ownership rolls back cleanly"
+fi
+stop_stub
+
+# The same cancellation after the launch Turn commits must preserve the
+# background job but release the frozen side of the published rotation.
+# Curses routes Ctrl-C through the normal UI cancellation handshake and exits
+# only after Coordinator has settled that ownership frame.
+if [ -x "$obj/ptydrive" ]; then
+ rm -rf "$HOME/.fugu"
+ rm -f "$dir/reqout" "$dir/reqlog" "$dir/rotation-cursesout"
+ start_stub "BGTOOL OK JOBOUTPUT"
+ write_conf "$stubport"
+ printf 'launch committed background\r\000\000\000poll committed background\r\000\003' | \
+ TERM=vt100 HOME=$HOME FUGU_SELFTEST=1 \
+ FUGU_ARTIFACT_FRAME0_DELAY_MS=3000 \
+ FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \
+ "$obj/ptydrive" "$fugu" -d \
+ >"$dir/rotation-cursesout" 2>&1
+ rc=$?
+ ok "background rotation cancel: curses exits after Ctrl-C" \
+ $([ "$rc" -eq 0 ]; echo $?)
+ n=$(grep -h -c '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null | \
+ awk '{ n += $1 } END { print n + 0 }')
+ ok "background rotation cancel: launch Turn remains committed" \
+ $([ "$n" -eq 1 ]; echo $?)
+ n=$(grep -h -c '"outcome":"abandoned"' \
+ "$sessions"/*.ndjson 2>/dev/null | \
+ awk '{ n += $1 } END { print n + 0 }')
+ ok "background rotation cancel: poll Turn is abandoned once" \
+ $([ "$n" -eq 1 ]; echo $?)
+ n=$(grep -Fc 'selftest frame0 Artifact rotation published' \
+ "$dir/rotation-cursesout" 2>/dev/null)
+ ok "background rotation cancel: committed frame publishes once" \
+ $([ "$n" -eq 1 ]; echo $?)
+ n=$(grep -Fc 'selftest Artifact RELEASE job=1' \
+ "$dir/rotation-cursesout" 2>/dev/null)
+ ok "background rotation cancel: committed frozen spool releases once" \
+ $([ "$n" -eq 1 ]; echo $?)
+ if grep -Eq 'selftest Artifact CANCEL_ACK|bad Artifact|mismatched Artifact|stale Artifact|missing cancelled Artifact' \
+ "$dir/rotation-cursesout" 2>/dev/null; then
+ fail "background rotation cancel: committed ownership settles cleanly"
+ else
+ pass "background rotation cancel: committed ownership settles cleanly"
+ fi
+ stop_stub
+else
+ fail "background rotation cancel: ptydrive helper not built"
+fi
+
+# Force a completed committed job into normal archival while its Artifact
+# terminal is delayed and converted to STORE_ERROR. DROP is already queued
+# on the Tool socket when Coordinator sends CANCEL; the ordered Tool output
+# must therefore be DROP followed by the exact job-wide CANCEL_ACK.
+rm -rf "$HOME/.fugu"
+rm -f "$dir/out" "$dir/errout" "$dir/reqout"
+start_stub "BGFAST OK RETRY429 OK JOBOUTPUT OK"
+write_conf "$stubport"
+printf 'launch short background\nwait for store failure\nprobe cancelled background\n/quit\n' \
+ > "$dir/lin"
+FUGU_SELFTEST=1 FUGU_ARTIFACT_ARCHIVE_COMPLETED=1 \
+ FUGU_ARTIFACT_DROP_DELAY_MS=1500 \
+ FUGU_ARTIFACT_STORE_ERROR_DELAY_MS=500 \
+ FUGU_CONF=$dir/fugu.conf FUGU_LIBEXEC=$dir/libexec/fugu \
+ FUGU_CA_FILE=$dir/cert.pem FUGU_RETRY_BASE_MS=50 \
+ "$fugu" -d <"$dir/lin" >"$dir/out" 2>"$dir/errout"
+rc=$?
+ok "background retirement race: session completes" \
+ $([ "$rc" -eq 0 ]; echo $?)
+n=$(grep -h -c '"outcome":"ok"' "$sessions"/*.ndjson 2>/dev/null | \
+ awk '{ n += $1 } END { print n + 0 }')
+ok "background retirement race: all three Turns commit" \
+ $([ "$n" -eq 3 ]; echo $?)
+grep -q 'shell_output: no such job 1' "$dir/reqout" 2>/dev/null
+ok "background retirement race: failed job ownership is gone" $?
+drop_line=$(grep -n -m 1 'selftest Artifact DROP queued' \
+ "$dir/errout" 2>/dev/null | cut -d: -f1)
+error_line=$(grep -n -m 1 'selftest STORE_ERROR injected' \
+ "$dir/errout" 2>/dev/null | cut -d: -f1)
+ack_line=$(grep -n -m 1 'selftest Artifact CANCEL_ACK job=1' \
+ "$dir/errout" 2>/dev/null | cut -d: -f1)
+[ -n "$drop_line" ] && [ -n "$error_line" ] && [ -n "$ack_line" ] && \
+ [ "$drop_line" -lt "$error_line" ] && [ "$error_line" -lt "$ack_line" ]
+ok "background retirement race: DROP precedes STORE_ERROR and CANCEL_ACK" $?
+n=$(grep -Ec 'selftest Artifact DROP queued|selftest STORE_ERROR injected|selftest Artifact CANCEL_ACK job=1' \
+ "$dir/errout" 2>/dev/null)
+ok "background retirement race: each protocol trace occurs exactly once" \
+ $([ "$n" -eq 3 ]; echo $?)
+if grep -Eq 'bad Artifact|mismatched Artifact|stale Artifact|unexpected Artifact|worker .* exited' \
+ "$dir/errout" 2>/dev/null; then
+ fail "background retirement race: exact-key composition stays valid"
+else
+ pass "background retirement race: exact-key composition stays valid"
+fi
+stop_stub
+
echo "$checks checks, $fails failures"
[ "$fails" -eq 0 ]
blob - 2f966dd42bec9929878d783035555b9a72ec860e
blob + 4c738bf1b95897ccc589f844b521cd6d3f82746d
--- regress/turn/stub.c
+++ regress/turn/stub.c
* stream with no content blocks), BADARGS (a complete but malformed
* subagent tool input), PROTODRAIN (a malformed Generation followed by a
* delayed Provider terminal), NOARGS (a subagent tool call with absent input),
- * WEBFETCH (a brokered web_fetch),
- * BGTOOL (shell_bg sleep 300), ERROR_BG (the provider failure after
+ * WEBFETCH (a brokered web_fetch), OUTPUTREAD (an explicit brokered Artifact
+ * page), OUTPUTREADDEFAULT (a defaults-only Artifact page),
+ * BGTOOL (shell_bg sleep 300), BGSHORT (shell_bg sleep 1),
+ * BGFAST (shell_bg immediate output),
+ * ERROR_BG (the provider failure after
* that launch), JOBOUTPUT (shell_output job 1), DOUBLE (a hostile error event
* appended after a complete OK stream), AGENTCALL (lead delegates to a
* labelled subagent), AGENTREAD (the subagent reads a fixture), AGENTREPORT
"event: message_stop\n"
"data: {\"type\":\"message_stop\"}\n\n";
+/* A printable result just over the provider-visible ceiling must be replaced
+ * by Fugu's abbreviated Artifact document before the next Generation. */
+static const char sse_artifact_tool[] =
+"event: message_start\n"
+"data: {\"type\":\"message_start\",\"message\":{\"usage\":"
+ "{\"input_tokens\":9,\"output_tokens\":1}}}\n\n"
+"event: content_block_start\n"
+"data: {\"type\":\"content_block_start\",\"index\":0,"
+ "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_artifact\","
+ "\"name\":\"shell\"}}\n\n"
+"event: content_block_delta\n"
+"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":"
+ "{\"type\":\"input_json_delta\",\"partial_json\":"
+ "\"{\\\"command\\\":\\\"perl -e 'print q(ARTIFACT-MARKER); "
+ "print q(x) x 65521; print pack(q(C*),65,0,255,66); "
+ "print q(ARTIFACT-MARKER) "
+ "x 24000'\\\"}\"}}\n\n"
+"event: content_block_stop\n"
+"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n"
+"event: message_delta\n"
+"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":"
+ "\"tool_use\"},\"usage\":{\"output_tokens\":12}}\n\n"
+"event: message_stop\n"
+"data: {\"type\":\"message_stop\"}\n\n";
+
/* The lead delegates a self-contained task to a labelled subagent. */
static const char sse_agentcall[] =
"event: message_start\n"
send_webfetch_path(conn, port, "web");
}
+static int
+hex_lower(int ch)
+{
+ return ((ch >= '0' && ch <= '9') || (ch >= 'a' && ch <= 'f'));
+}
+
static void
+output_read_id(const char *req, char id[33])
+{
+ static const char marker[] = "\\\"id\\\":\\\"";
+ const char *p;
+ size_t i;
+
+ p = strstr(req, marker);
+ if (p == NULL)
+ errx(1, "OUTPUTREAD fixture missing Artifact id");
+ p += sizeof(marker) - 1;
+ for (i = 0; i < 32; i++) {
+ if (!hex_lower((unsigned char)p[i]))
+ errx(1, "OUTPUTREAD fixture malformed Artifact id");
+ id[i] = p[i];
+ }
+ id[32] = '\0';
+ if (p[32] != '\\' || p[33] != '"')
+ errx(1, "OUTPUTREAD fixture unterminated Artifact id");
+}
+
+/* Recover the opaque id from the preceding generated Artifact result, then
+ * ask the Coordinator-local output_read tool for an exact binary page. */
+static void
+send_output_read(struct tls *conn, const char *req)
+{
+ char id[33], sse[2048];
+ int n;
+
+ output_read_id(req, id);
+
+ n = snprintf(sse, sizeof(sse),
+ "event: message_start\n"
+ "data: {\"type\":\"message_start\",\"message\":{\"usage\":"
+ "{\"input_tokens\":9,\"output_tokens\":1}}}\n\n"
+ "event: content_block_start\n"
+ "data: {\"type\":\"content_block_start\",\"index\":0,"
+ "\"content_block\":{\"type\":\"tool_use\",\"id\":"
+ "\"toolu_output_read\",\"name\":\"output_read\"}}\n\n"
+ "event: content_block_delta\n"
+ "data: {\"type\":\"content_block_delta\",\"index\":0,"
+ "\"delta\":{\"type\":\"input_json_delta\","
+ "\"partial_json\":\"{\\\"id\\\":\\\"%s\\\","
+ "\\\"offset\\\":65536,\\\"limit\\\":4,"
+ "\\\"encoding\\\":\\\"base64\\\"}\"}}\n\n"
+ "event: content_block_stop\n"
+ "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n"
+ "event: message_delta\n"
+ "data: {\"type\":\"message_delta\",\"delta\":{"
+ "\"stop_reason\":\"tool_use\"},\"usage\":{"
+ "\"output_tokens\":12}}\n\n"
+ "event: message_stop\n"
+ "data: {\"type\":\"message_stop\"}\n\n", id);
+ if (n < 0 || (size_t)n >= sizeof(sse))
+ errx(1, "OUTPUTREAD fixture too large");
+ (void)tls_write_all(conn, body_ok, strlen(body_ok));
+ write_chunk(conn, sse, (size_t)n);
+ (void)tls_write_all(conn, "0\r\n\r\n", 5);
+}
+
+/* Exercise Coordinator defaults with id as the only supplied argument. */
+static void
+send_output_read_default(struct tls *conn, const char *req)
+{
+ char id[33], sse[2048];
+ int n;
+
+ output_read_id(req, id);
+ n = snprintf(sse, sizeof(sse),
+ "event: message_start\n"
+ "data: {\"type\":\"message_start\",\"message\":{\"usage\":"
+ "{\"input_tokens\":9,\"output_tokens\":1}}}\n\n"
+ "event: content_block_start\n"
+ "data: {\"type\":\"content_block_start\",\"index\":0,"
+ "\"content_block\":{\"type\":\"tool_use\",\"id\":"
+ "\"toolu_output_read_default\",\"name\":\"output_read\"}}\n\n"
+ "event: content_block_delta\n"
+ "data: {\"type\":\"content_block_delta\",\"index\":0,"
+ "\"delta\":{\"type\":\"input_json_delta\","
+ "\"partial_json\":\"{\\\"id\\\":\\\"%s\\\"}\"}}\n\n"
+ "event: content_block_stop\n"
+ "data: {\"type\":\"content_block_stop\",\"index\":0}\n\n"
+ "event: message_delta\n"
+ "data: {\"type\":\"message_delta\",\"delta\":{"
+ "\"stop_reason\":\"tool_use\"},\"usage\":{"
+ "\"output_tokens\":12}}\n\n"
+ "event: message_stop\n"
+ "data: {\"type\":\"message_stop\"}\n\n", id);
+ if (n < 0 || (size_t)n >= sizeof(sse))
+ errx(1, "OUTPUTREADDEFAULT fixture too large");
+ (void)tls_write_all(conn, body_ok, strlen(body_ok));
+ write_chunk(conn, sse, (size_t)n);
+ (void)tls_write_all(conn, "0\r\n\r\n", 5);
+}
+
+static void
send_wave_webfetch(struct tls *conn, int port, const char *req)
{
if (strstr(req, "WAVE-PROMPT-A") != NULL)
"event: message_stop\n"
"data: {\"type\":\"message_stop\"}\n\n";
+/* A short job completes after its launching Turn can commit. A later
+ * retrying request keeps the Coordinator event loop alive long enough to
+ * exercise asynchronous current-spool retirement. */
+static const char sse_bgshort[] =
+"event: message_start\n"
+"data: {\"type\":\"message_start\",\"message\":{\"usage\":"
+ "{\"input_tokens\":9,\"output_tokens\":1}}}\n\n"
+"event: content_block_start\n"
+"data: {\"type\":\"content_block_start\",\"index\":0,"
+ "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_short\","
+ "\"name\":\"shell_bg\"}}\n\n"
+"event: content_block_delta\n"
+"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":"
+ "{\"type\":\"input_json_delta\",\"partial_json\":"
+ "\"{\\\"command\\\":\\\"sleep 1; echo short-done\\\"}\"}}\n\n"
+"event: content_block_stop\n"
+"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n"
+"event: message_delta\n"
+"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":"
+ "\"tool_use\"},\"usage\":{\"output_tokens\":12}}\n\n"
+"event: message_stop\n"
+"data: {\"type\":\"message_stop\"}\n\n";
+
+/* Complete comfortably before the retrying next Turn so its current spool
+ * can be archived without a model-requested shell_output poll. */
+static const char sse_bgfast[] =
+"event: message_start\n"
+"data: {\"type\":\"message_start\",\"message\":{\"usage\":"
+ "{\"input_tokens\":9,\"output_tokens\":1}}}\n\n"
+"event: content_block_start\n"
+"data: {\"type\":\"content_block_start\",\"index\":0,"
+ "\"content_block\":{\"type\":\"tool_use\",\"id\":\"toolu_fast\","
+ "\"name\":\"shell_bg\"}}\n\n"
+"event: content_block_delta\n"
+"data: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":"
+ "{\"type\":\"input_json_delta\",\"partial_json\":"
+ "\"{\\\"command\\\":\\\"echo fast-done\\\"}\"}}\n\n"
+"event: content_block_stop\n"
+"data: {\"type\":\"content_block_stop\",\"index\":0}\n\n"
+"event: message_delta\n"
+"data: {\"type\":\"message_delta\",\"delta\":{\"stop_reason\":"
+ "\"tool_use\"},\"usage\":{\"output_tokens\":12}}\n\n"
+"event: message_stop\n"
+"data: {\"type\":\"message_stop\"}\n\n";
+
/* Probe the exact id allocated by the preceding BGTOOL response. If the
* failed turn was rolled back, shell_output returns "no such job 1" and
* that tool error appears in the following request. */
write_chunk(conn, sse_tooluse,
strlen(sse_tooluse));
(void)tls_write_all(conn, "0\r\n\r\n", 5);
+ } else if (strcmp(argv[i], "ARTIFACTTOOL") == 0) {
+ (void)tls_write_all(conn, body_ok,
+ strlen(body_ok));
+ write_chunk(conn, sse_artifact_tool,
+ strlen(sse_artifact_tool));
+ (void)tls_write_all(conn, "0\r\n\r\n", 5);
+ } else if (strcmp(argv[i], "OUTPUTREAD") == 0) {
+ send_output_read(conn, req);
+ } else if (strcmp(argv[i], "OUTPUTREADDEFAULT") == 0) {
+ send_output_read_default(conn, req);
} else if (strcmp(argv[i], "AGENTCALL") == 0) {
(void)tls_write_all(conn, body_ok,
strlen(body_ok));
write_chunk(conn, sse_bgtool,
strlen(sse_bgtool));
(void)tls_write_all(conn, "0\r\n\r\n", 5);
+ } else if (strcmp(argv[i], "BGSHORT") == 0) {
+ (void)tls_write_all(conn, body_ok,
+ strlen(body_ok));
+ write_chunk(conn, sse_bgshort,
+ strlen(sse_bgshort));
+ (void)tls_write_all(conn, "0\r\n\r\n", 5);
+ } else if (strcmp(argv[i], "BGFAST") == 0) {
+ (void)tls_write_all(conn, body_ok,
+ strlen(body_ok));
+ write_chunk(conn, sse_bgfast,
+ strlen(sse_bgfast));
+ (void)tls_write_all(conn, "0\r\n\r\n", 5);
} else if (strcmp(argv[i], "ERROR_BG") == 0) {
(void)tls_write_all(conn, body_ok,
strlen(body_ok));
blob - 017c177fb52c67a49420620a94e387cd98208b87
blob + c41d37e01f962721f2c0d9941f3795cbc491b823
--- regress/web/tooldefs_test.c
+++ regress/web/tooldefs_test.c
memset(&cf, 0, sizeof(cf));
defs = tooldefs(&cf, NULL, &n);
+ d = find(defs, n, "output_read");
+ CHECK(d != NULL && strstr(d->schema, "\"required\":[\"id\"]") != NULL);
+ CHECK(d != NULL && strstr(d->schema, "\"offset\"") != NULL &&
+ strstr(d->schema, "\"limit\"") != NULL &&
+ strstr(d->schema, "\"encoding\"") != NULL &&
+ strstr(d->schema, "\"path\"") == NULL);
CHECK(find(defs, n, "web_search") == NULL);
CHECK(find(defs, n, "web_fetch") == NULL);
CHECK(find(defs, n, "http_request") == NULL);
blob - a0d582a0446f829a8eb7822ad01aaecd3936273e
blob + 700421220e2a3822ce1396669b4f4dce9a7609e1
--- scripts/live-check
+++ scripts/live-check
"root:_fugu:2555" ]
ok "installed front binary is root:_fugu mode 2555" $?
- workers='fugu-api fugu-editor fugu-tool fugu-tty fugu-web'
+ workers='fugu-api fugu-artifact fugu-editor fugu-tool fugu-tty fugu-web'
rc=0
for worker in $workers; do
path=$prefix/libexec/fugu/$worker
"root:wheel:555" ] ||
rc=1
done
- ok "all five workers are root:wheel mode 0555" "$rc"
+ ok "all six workers are root:wheel mode 0555" "$rc"
[ -f "$prefix/man/man1/fugu.1" ] &&
[ -f "$prefix/man/man5/fugu.conf.5" ] &&
[ "$(stat -f '%Su:%Sg:%Lp' "$prefix/man/man1/fugu.1")" = \
blob - 3a5b3e5ba5f5dbdea6852e9508b54cb9e21709bb
blob + c805edb8d73ef9b945e5385a660518f1fa6fe68f
--- src/fugu/Makefile
+++ src/fugu/Makefile
generation.c compaction.c msg.c \
provider_state.c \
json.c utf8.c artifact_result.c artifact_page.c artifact_lock.c \
- model_window.c effort.c \
- buf.c imsgev.c log.c xmalloc.c
+ artifact_pipe.c artifact_scope.c model_window.c effort.c buf.c \
+ imsgev.c log.c xmalloc.c
MAN= fugu.1 fugu.conf.5
MANDIR= ${PREFIX}/man/man
blob - 7fad8eb0c201bda22f5fd69cfd1f0bd99e5a4c4c
blob + 627654da188367f06d91eed00db8e43157e03c0c
--- src/fugu/artifact_capture.c
+++ src/fugu/artifact_capture.c
memset(capture, 0, sizeof(*capture));
}
+size_t
+artifact_capture_read_limit(const struct artifact_capture *capture, size_t want)
+{
+ uint64_t left;
+
+ if (capture->observed >= (uint64_t)ARTIFACT_RAW_MAX + 1)
+ return (0);
+ left = (uint64_t)ARTIFACT_RAW_MAX + 1 - capture->observed;
+ if ((uint64_t)want > left)
+ want = (size_t)left;
+ return (want);
+}
+
int
artifact_capture_add(struct artifact_capture *capture, const void *data,
size_t len)
blob - 3759b94c1981bf6a95b6862330ebfc0e7a7fe3a2
blob + 6d043e6f5cf6d3c9eec713aeeca0feef479f5972
--- src/fugu/artifact_capture.h
+++ src/fugu/artifact_capture.h
void artifact_capture_init(struct artifact_capture *);
void artifact_capture_reset(struct artifact_capture *);
void artifact_capture_free(struct artifact_capture *);
+/* Bound one producer read to the first byte that proves a hard-cap crossing. */
+size_t artifact_capture_read_limit(const struct artifact_capture *, size_t);
/* Adds observed bytes, retaining at most 16 MiB. Returns 1 on cap crossing. */
int artifact_capture_add(struct artifact_capture *, const void *, size_t);
void artifact_capture_view(const struct artifact_capture *,
blob - c32217e2d36f75fa53dfe11fdcd20543936bc89a
blob + 8be7494b9a5dec8d0f37df4612a802afac4bc1bf
--- src/fugu/artifact_lock.c
+++ src/fugu/artifact_lock.c
*/
#include <sys/types.h>
+#include <sys/file.h>
#include <sys/stat.h>
#include <fcntl.h>
(flags & O_ACCMODE) != O_RDWR ||
(flags & (O_APPEND | O_NONBLOCK)) != 0 ||
(fdflags = fcntl(fd, F_GETFD)) == -1 ||
- (fdflags & FD_CLOEXEC) == 0)
+ (fdflags & FD_CLOEXEC) == 0 ||
+ flock(fd, LOCK_EX | LOCK_NB) == -1)
return (0);
return (1);
}
blob - cba1d4dc9abc56081db28756f3f9181b58d24424
blob + 96c31c3b305d3ce2dcb11879f7b2effe2f43ba73
--- src/fugu/artifact_page.c
+++ src/fugu/artifact_page.c
if (encoding == ARTIFACT_PAGE_TEXT) {
while (used < len) {
unit = escaped_unit(raw + used, len - used, &nraw);
- if (unit == 0) {
- *retry_base64 = 1;
- return (retry_result(out, id, offset));
+ if (unit == 0 ||
+ (unit == SIZE_MAX && offset + len == total)) {
+ if (used == 0) {
+ *retry_base64 = 1;
+ return (retry_result(out, id, offset));
+ }
+ break;
}
- if (unit == SIZE_MAX && offset + len == total) {
- *retry_base64 = 1;
- return (retry_result(out, id, offset));
- }
if (unit == SIZE_MAX || unit >
ARTIFACT_PAGE_DATA_ENCODED_MAX - enc)
break;
blob - /dev/null
blob + 123933b8353f2b99060d6a3f071750b22d0ad4c5 (mode 644)
--- /dev/null
+++ src/fugu/artifact_pipe.c
+/*
+ * Copyright (c) 2026 Isaac <isaac@itm.works>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+
+#include <fcntl.h>
+#include <unistd.h>
+
+#include "artifact_pipe.h"
+
+int
+artifact_producer_endpoint_valid(int fd)
+{
+ struct stat st;
+ int flags, fdflags;
+
+ /* OpenBSD's bidirectional pipe extension reports both pipe(2)
+ * endpoints as exactly O_RDWR. Protocol assigns this endpoint only
+ * the producer direction; it must also remain a blocking FIFO and
+ * close across exec. */
+ return (fd >= 0 && fstat(fd, &st) == 0 && S_ISFIFO(st.st_mode) &&
+ (flags = fcntl(fd, F_GETFL)) != -1 && flags == O_RDWR &&
+ (fdflags = fcntl(fd, F_GETFD)) != -1 &&
+ (fdflags & FD_CLOEXEC) != 0);
+}
blob - /dev/null
blob + ffeec137dd2076e7b78851f5e643a924b9c2d11f (mode 644)
--- /dev/null
+++ src/fugu/artifact_pipe.h
+/*
+ * Copyright (c) 2026 Isaac <isaac@itm.works>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ */
+
+#ifndef ARTIFACT_PIPE_H
+#define ARTIFACT_PIPE_H
+
+/* Exact validation for the producer endpoint brokered by Artifact. */
+int artifact_producer_endpoint_valid(int);
+
+#endif /* ARTIFACT_PIPE_H */
blob - 444af14d763fae39f59299bfcb868bf57892a4d5
blob + c121936fd24e3eb1237528dc5cba5d5fded4e091
--- src/fugu/coord.c
+++ src/fugu/coord.c
*/
#include <sys/types.h>
+#include <sys/queue.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <glob.h>
#include <imsg.h>
+#include <limits.h>
#include <signal.h>
#include <stdarg.h>
#include <stdint.h>
#include "artifact_result.h"
#include "artifact_page.h"
#include "artifact_lock.h"
+#include "artifact_pipe.h"
+#include "artifact_scope.h"
#define TURN_ITER_MAX 50 /* behavior.md section 5 */
#define MAX_TOOL_CALLS GENERATION_CALL_MAX
struct coord;
struct artifact_live {
- struct artifact_live *next;
+ LIST_ENTRY(artifact_live) entry;
uint32_t correlation;
+ unsigned int refs;
+ int removed;
int background;
int adopted;
+ int job_id;
+ int job_failed;
+ int failed_discarded;
int turn_owned;
int discard_when_terminal;
+ int release_when_terminal;
+ int rotation_done;
+ int rotation_adopted;
+ int rotation_detach;
+ int drop_seen;
+ int drop_pending;
+ int cancel_hold;
+ int cancel_retire;
+ int cancel_pending;
+ uint32_t frozen_correlation;
+ char frozen_id[FUGU_ARTIFACT_ID_BUFSZ];
int stream_done;
int terminal_done;
struct artifact_stream stream;
struct artifact_terminal terminal;
};
+LIST_HEAD(artifact_live_head, artifact_live);
+
/* One on-demand fugu-tool -r instance. Its conversation and local tool
* exchanges stay inside that process; only the final report is retained. */
struct subagent {
uint32_t artifact_ctl_expected;
int artifact_ctl_done;
uint32_t artifact_call_id;
+ int artifact_call_job_id;
+ int artifact_call_shell_bg;
+ int artifact_call_shell_output;
struct artifact_live *artifact_call;
struct artifact_live *artifact_wait;
- struct artifact_live *artifacts;
+ struct artifact_live_head artifacts;
struct buf artifact_read_data;
struct artifact_read_result artifact_read_result;
struct artifact_read_req artifact_read_req;
enum turn_tools_mode, int);
static enum turn_io compact_projection(struct coord *, int, int,
enum turn_tools_mode, struct generation_usage *, char *, size_t);
+static void artifact_background_failure(struct coord *,
+ struct artifact_live *);
+static void artifact_rotation_event(struct coord *, uint32_t,
+ const u_char *, size_t);
+static void artifact_drop_event(struct coord *, uint32_t,
+ const u_char *, size_t);
+static void artifact_cancel_ack_event(struct coord *, uint32_t,
+ const u_char *, size_t);
+static void artifact_cancelled_rotation(struct coord *,
+ struct artifact_live *);
+static void artifact_discard(struct coord *, const struct artifact_live *);
+static void artifact_job_control(struct coord *, uint32_t,
+ const struct artifact_live *);
+static void artifact_cancel_retire(struct coord *, int);
+static int artifact_cancelled_tool_result_valid(struct coord *,
+ const struct artifact_live *, const struct tool_result *);
static struct subagent *
agent_by_api_id(struct coord *c, uint32_t id)
{
struct artifact_live *live;
- for (live = c->artifacts; live != NULL; live = live->next)
+ LIST_FOREACH(live, &c->artifacts, entry)
if (live->correlation == correlation)
return (live);
return (NULL);
if (!artifact_id_wire_valid(id))
return (NULL);
- for (live = c->artifacts; live != NULL; live = live->next)
+ LIST_FOREACH(live, &c->artifacts, entry)
if (live->stream_done &&
live->stream.status == ARTIFACT_STREAM_AVAILABLE &&
strcmp(live->stream.id, id) == 0)
return (NULL);
}
+static int
+artifact_result_names_call_stream(const struct tool_result *result,
+ const struct artifact_live *live)
+{
+ static const char zero[FUGU_ARTIFACT_ID_BUFSZ];
+
+ if (live == NULL || !live->stream_done)
+ return (0);
+ if (live->stream.status == ARTIFACT_STREAM_AVAILABLE)
+ return (artifact_id_wire_valid(result->artifact_id) &&
+ strcmp(result->artifact_id, live->stream.id) == 0);
+ if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE)
+ return (memcmp(result->artifact_id, zero, sizeof(zero)) == 0);
+ return (0);
+}
+
+static int
+artifact_result_fields_zero(const struct tool_result *result)
+{
+ static const char zero[FUGU_ARTIFACT_ID_BUFSZ];
+
+ return (!result->artifact && !result->keep_artifact &&
+ !result->artifact_cap_reached && !result->spool_frozen &&
+ !result->spool_adopted && !result->spool_detach &&
+ result->background_job_id == 0 &&
+ result->termination == ARTIFACT_TERM_EXIT &&
+ result->observed_bytes == 0 && result->retained_bytes == 0 &&
+ memcmp(result->artifact_id, zero, sizeof(zero)) == 0);
+}
+
+static int
+artifact_result_matches_rotation(const struct tool_result *result,
+ const struct artifact_live *live)
+{
+ return (live != NULL && live->rotation_done && result->spool_frozen &&
+ result->spool_adopted == live->rotation_adopted &&
+ result->spool_detach == live->rotation_detach &&
+ result->background_job_id == live->job_id &&
+ memcmp(result->artifact_id, live->frozen_id,
+ sizeof(live->frozen_id)) == 0);
+}
+
static struct artifact_live *
artifact_live_new(struct coord *c, uint32_t correlation, int background)
{
live->correlation = correlation;
live->background = background;
live->turn_owned = 1;
- live->next = c->artifacts;
- c->artifacts = live;
+ LIST_INSERT_HEAD(&c->artifacts, live, entry);
return (live);
}
static void
+artifact_live_pin(struct artifact_live *live)
+{
+ if (live == NULL || live->refs == UINT_MAX)
+ fatalx("invalid Artifact live pin");
+ live->refs++;
+}
+
+static void
+artifact_live_unpin(struct artifact_live *live)
+{
+ if (live == NULL || live->refs == 0)
+ fatalx("invalid Artifact live unpin");
+ live->refs--;
+ if (live->removed && live->refs == 0)
+ free(live);
+}
+
+static void
+artifact_live_remove_now(struct coord *c, struct artifact_live *live)
+{
+ struct artifact_live *scan;
+
+ if (live == NULL || live->removed)
+ fatalx("duplicate Artifact live removal");
+ LIST_FOREACH(scan, &c->artifacts, entry)
+ if (scan == live)
+ break;
+ if (scan == NULL)
+ fatalx("lost Artifact live state");
+ LIST_FOREACH(scan, &c->artifacts, entry) {
+ if (scan == live || scan->frozen_correlation !=
+ live->correlation)
+ continue;
+ scan->frozen_correlation = 0;
+ }
+ LIST_REMOVE(live, entry);
+ live->removed = 1;
+ if (c->artifact_call == live)
+ c->artifact_call = NULL;
+ if (c->artifact_wait == live)
+ c->artifact_wait = NULL;
+ if (live->refs == 0)
+ free(live);
+}
+
+static void
artifact_live_remove(struct coord *c, struct artifact_live *live)
{
- struct artifact_live **link;
-
- for (link = &c->artifacts; *link != NULL; link = &(*link)->next) {
- if (*link != live)
- continue;
- *link = live->next;
+ if (live == NULL || live->removed)
+ fatalx("duplicate Artifact live removal");
+ if (live->cancel_hold) {
+ live->cancel_retire = 1;
if (c->artifact_call == live)
c->artifact_call = NULL;
- free(live);
+ if (c->artifact_wait == live)
+ c->artifact_wait = NULL;
return;
}
- fatalx("lost Artifact live state");
+ artifact_live_remove_now(c, live);
}
static uint32_t
struct artifact_live *live;
artifact_turn_ctl(c, FUGU_IMSG_ARTIFACT_TURN_COMMIT);
- for (live = c->artifacts; live != NULL; live = live->next)
+ LIST_FOREACH(live, &c->artifacts, entry)
if (live->turn_owned)
live->turn_owned = 0;
c->nturn_artifacts = 0;
struct artifact_live *live, *next;
artifact_turn_ctl(c, FUGU_IMSG_ARTIFACT_TURN_ABORT);
- for (live = c->artifacts; live != NULL; live = next) {
- next = live->next;
+ LIST_FOREACH_SAFE(live, &c->artifacts, entry, next) {
if (live->turn_owned)
artifact_live_remove(c, live);
}
}
static int
+path_contains(const char *parent, const char *child)
+{
+ size_t len;
+
+ len = strlen(parent);
+ if (len == 1 && parent[0] == '/')
+ return (child[0] == '/');
+ return (strncmp(parent, child, len) == 0 &&
+ (child[len] == '\0' || child[len] == '/'));
+}
+
+static int
+paths_overlap(const char *a, const char *b)
+{
+ return (path_contains(a, b) || path_contains(b, a));
+}
+
+/*
+ * The persistent Lead Tool unveils the physical project tree plus these
+ * fixed runtime roots. Managed Artifact bytes must be outside every one of
+ * those trees: a broader unveil cannot be repaired after a Tool starts, and
+ * the Tool Role must never receive raw-store path authority.
+ */
+static void
+artifact_root_scope_check(const struct coord *c, const char *root)
+{
+ static const char *const fixed[] = {
+ "/dev/null", "/usr", "/bin", "/sbin", "/lib", "/libexec"
+ };
+ static const char *const network[] = {
+ "/etc/resolv.conf", "/etc/hosts", "/etc/ssl"
+ };
+ char path[PATH_MAX];
+ size_t i;
+
+ if (realpath(".", path) == NULL)
+ fatal("realpath project directory");
+ if (paths_overlap(root, path))
+ fatalx("managed Artifact storage overlaps Tool filesystem authority");
+ for (i = 0; i < sizeof(fixed) / sizeof(fixed[0]); i++) {
+ if (realpath(fixed[i], path) == NULL &&
+ (errno != ENOENT || strlcpy(path, fixed[i], sizeof(path)) >=
+ sizeof(path)))
+ fatal("resolve Tool runtime path %s", fixed[i]);
+ if (paths_overlap(root, path))
+ fatalx("managed Artifact storage overlaps Tool filesystem authority");
+ }
+ if (!c->conf->allow_subprocess_net)
+ return;
+ for (i = 0; i < sizeof(network) / sizeof(network[0]); i++) {
+ if (realpath(network[i], path) == NULL &&
+ (errno != ENOENT || strlcpy(path, network[i], sizeof(path)) >=
+ sizeof(path)))
+ fatal("resolve Tool network path %s", network[i]);
+ if (paths_overlap(root, path))
+ fatalx("managed Artifact storage overlaps Tool filesystem authority");
+ }
+}
+
+/* Resolve HOME and ~/.fugu before any worker, Journal, or Provider action.
+ * The preliminary physical-HOME check catches fixed-root HOME values before
+ * creating ~/.fugu; the second catches a pre-existing ~/.fugu symlink. */
+static void
+artifact_root_prepare(struct coord *c)
+{
+ struct stat st;
+ const char *home;
+ char fugu[PATH_MAX], physical[PATH_MAX], root[PATH_MAX];
+
+ home = getenv("HOME");
+ if (home == NULL || home[0] != '/' || realpath(home, physical) == NULL ||
+ (size_t)snprintf(root, sizeof(root), "%s/.fugu/artifacts",
+ physical) >= sizeof(root))
+ fatalx("cannot use managed Artifact storage");
+ artifact_root_scope_check(c, root);
+ if ((size_t)snprintf(fugu, sizeof(fugu), "%s/.fugu", home) >=
+ sizeof(fugu) || (mkdir(fugu, 0700) == -1 && errno != EEXIST) ||
+ realpath(fugu, physical) == NULL ||
+ (size_t)snprintf(root, sizeof(root), "%s/artifacts", physical) >=
+ sizeof(root))
+ fatalx("cannot use managed Artifact storage");
+ artifact_root_scope_check(c, root);
+ if (stat(physical, &st) == -1 || !S_ISDIR(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0700 ||
+ strlcpy(c->artifact_root, root, sizeof(c->artifact_root)) >=
+ sizeof(c->artifact_root))
+ fatalx("cannot use managed Artifact storage");
+}
+
+static int
protect_has(const struct buf *b, const char *path)
{
size_t off, len;
p = payload.data;
len = payload.len;
switch (imsg_get_type(imsg)) {
+ case FUGU_IMSG_TOOL_ARTIFACT_ROTATE:
+ artifact_rotation_event(c, imsg_get_id(imsg), p, len);
+ return;
case FUGU_IMSG_TOOL_OUT:
case FUGU_IMSG_TOOL_DIFF:
if (p == NULL || len == 0)
fatalx("bad cancelled Tool chunk");
return;
case FUGU_IMSG_TOOL_RESULT:
- if (p == NULL || len != sizeof(struct tool_result))
+ {
+ struct artifact_live *call_live = c->artifact_call;
+ struct tool_result r;
+ int job_id;
+
+ if (call_live == NULL && c->artifact_call_id != 0)
+ call_live = artifact_live_by_correlation(c,
+ c->artifact_call_id);
+ job_id = call_live != NULL ? call_live->job_id : 0;
+ if (p == NULL || len != sizeof(r))
fatalx("bad cancelled Tool terminal");
+ memcpy(&r, p, sizeof(r));
+ if ((r.is_error != 0 && r.is_error != 1) ||
+ (r.artifact != 0 && r.artifact != 1) ||
+ (r.keep_artifact != 0 && r.keep_artifact != 1) ||
+ (r.artifact_cap_reached != 0 &&
+ r.artifact_cap_reached != 1) ||
+ (r.spool_frozen != 0 && r.spool_frozen != 1) ||
+ (r.spool_adopted != 0 && r.spool_adopted != 1) ||
+ (r.spool_detach != 0 && r.spool_detach != 1) ||
+ (r.spool_detach && !r.spool_adopted) ||
+ r.background_job_id < 0 ||
+ !artifact_cancelled_tool_result_valid(c, call_live, &r))
+ fatalx("mismatched cancelled Tool terminal");
+ artifact_cancelled_rotation(c, call_live);
c->tool_cancel_draining = 0;
c->tool_cancel_id = 0;
+ c->artifact_call = NULL;
+ c->artifact_call_id = 0;
+ c->artifact_call_job_id = 0;
+ c->artifact_call_shell_bg = 0;
+ c->artifact_call_shell_output = 0;
+ if (job_id > 0)
+ artifact_cancel_retire(c, job_id);
return;
+ }
default:
fatalx("bad cancelled Tool frame");
}
len = payload.len;
switch (imsg_get_type(imsg)) {
+ case FUGU_IMSG_TOOL_ARTIFACT_ROTATE:
+ artifact_rotation_event(c, imsg_get_id(imsg), p, len);
+ break;
case FUGU_IMSG_TOOL_OUT:
{
size_t max = c->artifact_call_id != 0 ?
(r.keep_artifact != 0 && r.keep_artifact != 1) ||
(r.artifact_cap_reached != 0 &&
r.artifact_cap_reached != 1) ||
+ (r.spool_frozen != 0 && r.spool_frozen != 1) ||
(r.spool_adopted != 0 && r.spool_adopted != 1) ||
(r.spool_detach != 0 && r.spool_detach != 1) ||
(r.spool_detach && !r.spool_adopted) ||
+ (!c->artifact_call_shell_output && r.spool_frozen) ||
+ (c->artifact_call_shell_output &&
+ (c->artifact_call == NULL || r.spool_frozen !=
+ c->artifact_call->rotation_done ||
+ (r.spool_frozen &&
+ !artifact_result_matches_rotation(&r,
+ c->artifact_call)) ||
+ (!r.spool_frozen && (r.spool_adopted ||
+ r.spool_detach ||
+ !artifact_result_names_call_stream(&r,
+ c->artifact_call))))) ||
+ r.background_job_id < 0 ||
(c->artifact_call_id == 0 &&
- (r.artifact || r.spool_adopted)))
+ !artifact_result_fields_zero(&r)))
c->turn_data_exceeded = 1;
else {
c->tool_is_error = r.is_error;
}
static int
-artifact_pipe_write_fd(int fd)
-{
- struct stat st;
- int flags, fdflags;
-
- return (fd >= 0 && fstat(fd, &st) == 0 && S_ISFIFO(st.st_mode) &&
- (flags = fcntl(fd, F_GETFL)) != -1 &&
- flags == O_WRONLY &&
- (fdflags = fcntl(fd, F_GETFD)) != -1 &&
- (fdflags & FD_CLOEXEC) != 0);
-}
-
-static int
artifact_id_zero(const char id[FUGU_ARTIFACT_ID_BUFSZ])
{
static const char zero[FUGU_ARTIFACT_ID_BUFSZ];
return (memcmp(id, zero, sizeof(zero)) == 0);
}
+static struct artifact_live *
+artifact_unavailable_by_job(struct coord *c, int job_id,
+ const struct artifact_live *exclude)
+{
+ struct artifact_live *live, *found = NULL;
+
+ LIST_FOREACH(live, &c->artifacts, entry) {
+ if (live == exclude || !live->background || !live->adopted ||
+ live->job_id != job_id || !live->stream_done ||
+ live->stream.status != ARTIFACT_STREAM_UNAVAILABLE)
+ continue;
+ if (found != NULL)
+ fatalx("ambiguous unavailable Artifact spool");
+ found = live;
+ }
+ return (found);
+}
+
+static struct artifact_live *
+artifact_rotation_frozen(struct coord *c, const struct artifact_live *live)
+{
+ struct artifact_live *frozen;
+
+ if (!live->rotation_done || live->frozen_correlation == 0 ||
+ (frozen = artifact_live_by_correlation(c,
+ live->frozen_correlation)) == NULL)
+ return (NULL);
+ if (frozen == live || !frozen->background || !frozen->adopted ||
+ frozen->job_id != live->job_id || !frozen->stream_done ||
+ (artifact_id_zero(live->frozen_id) &&
+ frozen->stream.status != ARTIFACT_STREAM_UNAVAILABLE) ||
+ (!artifact_id_zero(live->frozen_id) &&
+ (frozen->stream.status != ARTIFACT_STREAM_AVAILABLE ||
+ strcmp(frozen->stream.id, live->frozen_id) != 0)))
+ fatalx("lost frozen Artifact spool ownership");
+ return (frozen);
+}
+
+static int
+artifact_live_is_current(struct coord *c, const struct artifact_live *target)
+{
+ struct artifact_live *live;
+
+ LIST_FOREACH(live, &c->artifacts, entry)
+ if (live != target && live->background && live->adopted &&
+ live->job_id == target->job_id &&
+ live->frozen_correlation == target->correlation)
+ return (0);
+ return (1);
+}
+
+static int
+artifact_live_referenced_by_active(struct coord *c,
+ const struct artifact_live *target)
+{
+ struct artifact_live *live;
+
+ LIST_FOREACH(live, &c->artifacts, entry)
+ if (live != target && live->frozen_correlation ==
+ target->correlation && (live->refs != 0 ||
+ live == c->artifact_call || live == c->artifact_wait))
+ return (1);
+ return (0);
+}
+
static void
+artifact_cancel_retire(struct coord *c, int job_id)
+{
+ struct artifact_live *live, *next;
+
+ LIST_FOREACH_SAFE(live, &c->artifacts, entry, next) {
+ if (live->job_id != job_id || live->cancel_hold ||
+ !live->cancel_retire || live->refs != 0 ||
+ live == c->artifact_call || live == c->artifact_wait ||
+ artifact_live_referenced_by_active(c, live))
+ continue;
+ artifact_live_remove_now(c, live);
+ }
+}
+
+static void
+artifact_drop_ack(struct coord *c, const struct artifact_live *live)
+{
+ struct artifact_spool_drop drop;
+
+ memset(&drop, 0, sizeof(drop));
+ drop.job_id = (uint32_t)live->job_id;
+ if (live->stream.status == ARTIFACT_STREAM_AVAILABLE)
+ strlcpy(drop.artifact_id, live->stream.id,
+ sizeof(drop.artifact_id));
+ imsgev_send(&c->iev[ROLE_TOOL], FUGU_IMSG_TOOL_ARTIFACT_DROP_ACK,
+ live->correlation, &drop, sizeof(drop));
+}
+
+/*
+ * A spool swap changes ownership before the ordinary Tool result is
+ * discardable. Record that change as its own transaction frame so Turn
+ * cancellation cannot roll a committed job's replacement back into the
+ * aborted Turn.
+ */
+static void
+artifact_rotation_event(struct coord *c, uint32_t id, const u_char *p,
+ size_t len)
+{
+ struct artifact_spool_rotate rotate;
+ struct artifact_live *live, *frozen;
+ struct artifact_record record;
+
+ if (id == 0 || id != c->artifact_call_id || p == NULL ||
+ len != sizeof(rotate) ||
+ (live = c->artifact_call) == NULL || live->removed ||
+ live->correlation != id || !live->background ||
+ !live->stream_done || live->rotation_done ||
+ c->artifact_call_job_id <= 0)
+ fatalx("bad Artifact spool rotation");
+ memcpy(&rotate, p, sizeof(rotate));
+ if (rotate.job_id == 0 || rotate.job_id > INT_MAX ||
+ rotate.job_id != (uint32_t)c->artifact_call_job_id ||
+ rotate.replacement_adopted > 1 || rotate.detach > 1 ||
+ (!rotate.replacement_adopted &&
+ (rotate.detach || !artifact_id_zero(rotate.replacement_id))))
+ fatalx("invalid Artifact spool rotation state");
+ if (rotate.replacement_adopted &&
+ ((live->stream.status == ARTIFACT_STREAM_AVAILABLE &&
+ (!artifact_id_wire_valid(rotate.replacement_id) ||
+ strcmp(rotate.replacement_id, live->stream.id) != 0)) ||
+ (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ !artifact_id_zero(rotate.replacement_id))))
+ fatalx("mismatched Artifact spool replacement");
+
+ frozen = NULL;
+ if (artifact_id_wire_valid(rotate.frozen_id)) {
+ frozen = artifact_live_by_id(c, rotate.frozen_id);
+ if (frozen == NULL || frozen == live || !frozen->background ||
+ !frozen->adopted || frozen->job_id != (int)rotate.job_id ||
+ frozen->drop_seen ||
+ (rotate.replacement_adopted &&
+ rotate.detach != (uint32_t)!frozen->turn_owned))
+ fatalx("mismatched frozen Artifact spool");
+ } else if (!artifact_id_zero(rotate.frozen_id))
+ fatalx("malformed frozen Artifact spool id");
+ else {
+ /* An unavailable old reservation has no wire id, but its retained
+ * live record still proves job and Turn ownership. */
+ frozen = artifact_unavailable_by_job(c, (int)rotate.job_id,
+ live);
+ if (frozen == NULL || frozen->drop_seen ||
+ (rotate.replacement_adopted &&
+ rotate.detach != (uint32_t)!frozen->turn_owned))
+ fatalx("missing unavailable Artifact spool ownership");
+ }
+ if (!artifact_live_is_current(c, frozen))
+ fatalx("historical Artifact spool cannot rotate again");
+
+ live->job_id = (int)rotate.job_id;
+ live->rotation_done = 1;
+ live->rotation_adopted = (int)rotate.replacement_adopted;
+ live->rotation_detach = (int)rotate.detach;
+ live->frozen_correlation = frozen->correlation;
+ memcpy(live->frozen_id, rotate.frozen_id, sizeof(live->frozen_id));
+ if (live->rotation_adopted)
+ live->adopted = 1;
+ if (live->rotation_detach) {
+ live->turn_owned = 0;
+ if (live->stream.status == ARTIFACT_STREAM_AVAILABLE) {
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, live->stream.id, sizeof(record.id));
+ imsgev_send(&c->iev[ROLE_ARTIFACT],
+ FUGU_IMSG_ARTIFACT_DETACH, live->correlation,
+ &record, sizeof(record));
+ }
+ }
+ if (live->rotation_adopted && live->terminal_done &&
+ live->terminal.status == ARTIFACT_STORE_ERROR)
+ artifact_background_failure(c, live);
+}
+
+/* A completed, unpolled job asks to retire only its current spool. Available
+ * spools wait for the store terminal before ACK; unavailable reservations have
+ * no Artifact-side terminal and can be acknowledged immediately. */
+static void
+artifact_drop_event(struct coord *c, uint32_t id, const u_char *p, size_t len)
+{
+ struct artifact_spool_drop drop;
+ struct artifact_live *live;
+
+ if (id == 0 || p == NULL || len != sizeof(drop) ||
+ (live = artifact_live_by_correlation(c, id)) == NULL ||
+ live->removed || !live->background || !live->adopted ||
+ !live->stream_done || (live->job_failed && !live->cancel_hold) ||
+ live->drop_seen ||
+ !artifact_live_is_current(c, live))
+ fatalx("bad Artifact spool retirement");
+ memcpy(&drop, p, sizeof(drop));
+ if (drop.job_id == 0 || drop.job_id > INT_MAX ||
+ live->job_id != (int)drop.job_id ||
+ (live->stream.status == ARTIFACT_STREAM_AVAILABLE &&
+ (!artifact_id_wire_valid(drop.artifact_id) ||
+ strcmp(drop.artifact_id, live->stream.id) != 0)) ||
+ (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
+ !artifact_id_zero(drop.artifact_id)))
+ fatalx("mismatched Artifact spool retirement");
+ live->drop_seen = 1;
+ if (live->job_failed) {
+ if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE) {
+ artifact_live_remove(c, live);
+ return;
+ }
+ if (!live->terminal_done) {
+ live->drop_pending = 1;
+ live->discard_when_terminal = 1;
+ return;
+ }
+ if (!live->failed_discarded) {
+ artifact_discard(c, live);
+ live->failed_discarded = 1;
+ }
+ artifact_live_remove(c, live);
+ return;
+ }
+ if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE) {
+ artifact_drop_ack(c, live);
+ artifact_live_remove(c, live);
+ return;
+ }
+ if (!live->terminal_done) {
+ live->drop_pending = 1;
+ live->release_when_terminal = 0;
+ return;
+ }
+ if (live->terminal.status == ARTIFACT_STORE_ERROR) {
+ artifact_background_failure(c, live);
+ return;
+ }
+ artifact_discard(c, live);
+ artifact_drop_ack(c, live);
+ artifact_live_remove(c, live);
+}
+
+static void
+artifact_cancel_ack_event(struct coord *c, uint32_t id, const u_char *p,
+ size_t len)
+{
+ struct artifact_job_cancel_ack ack;
+ struct artifact_live *live, *marker = NULL;
+
+ if (id == 0 || id > INT_MAX || p == NULL || len != sizeof(ack))
+ fatalx("bad Artifact cancellation acknowledgement");
+ memcpy(&ack, p, sizeof(ack));
+ if (ack.job_id != id || !artifact_id_wire_valid(ack.artifact_id))
+ fatalx("invalid Artifact cancellation acknowledgement");
+ LIST_FOREACH(live, &c->artifacts, entry) {
+ if (!live->cancel_pending || live->job_id != (int)id)
+ continue;
+ if (marker != NULL)
+ fatalx("ambiguous Artifact cancellation acknowledgement");
+ marker = live;
+ }
+ if (marker == NULL || marker->job_id != (int)id ||
+ marker->stream.status != ARTIFACT_STREAM_AVAILABLE ||
+ strcmp(marker->stream.id, ack.artifact_id) != 0)
+ fatalx("stale Artifact cancellation acknowledgement");
+ marker->cancel_pending = 0;
+ LIST_FOREACH(live, &c->artifacts, entry)
+ if (live->job_id == (int)id)
+ live->cancel_hold = 0;
+ artifact_cancel_retire(c, (int)id);
+}
+
+static void
artifact_event(struct coord *c, struct imsg *imsg)
{
struct imsgev_payload payload;
stream.status != ARTIFACT_STREAM_UNAVAILABLE) ||
(stream.status == ARTIFACT_STREAM_AVAILABLE &&
(!artifact_id_wire_valid(stream.id) ||
- !artifact_pipe_write_fd(fd))) ||
+ !artifact_producer_endpoint_valid(fd))) ||
(stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
(fd != -1 || !artifact_id_zero(stream.id)))) {
if (fd != -1)
}
case FUGU_IMSG_ARTIFACT_TERMINAL: {
struct artifact_live *live;
- struct artifact_record record;
+ int preserve, wake;
if (imsgev_get_payload(imsg, &payload) == -1 ||
payload.data == NULL ||
live->terminal.status != ARTIFACT_STORE_ERROR))
fatalx("invalid Artifact terminal state");
live->terminal_done = 1;
- if (live->terminal.status == ARTIFACT_STORE_ERROR) {
- if (live->background || live->adopted) {
- memset(&record, 0, sizeof(record));
- strlcpy(record.id, live->stream.id,
- sizeof(record.id));
- imsgev_send(&c->iev[ROLE_TOOL],
- FUGU_IMSG_TOOL_ARTIFACT_CANCEL, 0,
- &record, sizeof(record));
- } else if (live == c->artifact_call && !c->tool_done)
- (void)kill(c->kids[ROLE_TOOL].pid, SIGTERM);
+ wake = c->wait == WAIT_ARTIFACT_TERMINAL &&
+ c->artifact_wait == live;
+ if (live->drop_pending) {
+ if (live->terminal.status == ARTIFACT_STORE_ERROR) {
+ artifact_background_failure(c, live);
+ } else if (live->job_failed) {
+ if (!live->failed_discarded) {
+ artifact_discard(c, live);
+ live->failed_discarded = 1;
+ }
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ } else {
+ artifact_discard(c, live);
+ artifact_drop_ack(c, live);
+ live->drop_pending = 0;
+ artifact_live_remove(c, live);
+ }
+ if (wake)
+ event_loopexit(NULL);
+ return;
}
+ if (live->terminal.status == ARTIFACT_STORE_ERROR &&
+ live->background && live->adopted && live->job_id > 0) {
+ artifact_background_failure(c, live);
+ if (wake)
+ event_loopexit(NULL);
+ return;
+ }
+ if (live->terminal.status == ARTIFACT_STORE_ERROR &&
+ !live->background && live == c->artifact_call &&
+ !c->tool_done)
+ (void)kill(c->kids[ROLE_TOOL].pid, SIGUSR1);
if (live->discard_when_terminal) {
- memset(&record, 0, sizeof(record));
- record.retained_bytes = live->terminal.retained_bytes;
- strlcpy(record.id, live->terminal.id, sizeof(record.id));
- imsgev_send(&c->iev[ROLE_ARTIFACT],
- FUGU_IMSG_ARTIFACT_DISCARD, live->correlation,
- &record, sizeof(record));
- artifact_live_remove(c, live);
+ if (!live->failed_discarded) {
+ artifact_discard(c, live);
+ if (live->job_failed)
+ live->failed_discarded = 1;
+ }
+ if (live->release_when_terminal && !live->job_failed)
+ artifact_job_control(c,
+ FUGU_IMSG_TOOL_ARTIFACT_RELEASE, live);
+ live->discard_when_terminal = 0;
+ live->release_when_terminal = 0;
+ preserve = live == c->artifact_call ||
+ live == c->artifact_wait;
+ if (!preserve)
+ artifact_live_remove(c, live);
+ if (wake)
+ event_loopexit(NULL);
return;
}
- if (c->wait == WAIT_ARTIFACT_TERMINAL &&
- c->artifact_wait == live)
+ if (wake)
event_loopexit(NULL);
return;
}
case FUGU_IMSG_ARTIFACT_DATA:
- if (c->wait != WAIT_ARTIFACT_READ || id == 0 ||
+ if (id == 0 || c->artifact_read_id == 0 ||
id != c->artifact_read_id || c->artifact_read_done ||
imsgev_get_payload(imsg, &payload) == -1 ||
payload.data == NULL || payload.len == 0 ||
buf_add(&c->artifact_read_data, payload.data, payload.len);
return;
case FUGU_IMSG_ARTIFACT_READ_RESULT:
- if (c->wait != WAIT_ARTIFACT_READ || id == 0 ||
+ if (id == 0 || c->artifact_read_id == 0 ||
id != c->artifact_read_id || c->artifact_read_done ||
imsgev_get_payload(imsg, &payload) == -1 ||
payload.data == NULL ||
case FUGU_IMSG_TOOL_OUT:
case FUGU_IMSG_TOOL_DIFF:
case FUGU_IMSG_TOOL_RESULT:
+ case FUGU_IMSG_TOOL_ARTIFACT_ROTATE:
+ case FUGU_IMSG_TOOL_ARTIFACT_DROP:
+ case FUGU_IMSG_TOOL_ARTIFACT_CANCEL_ACK:
case FUGU_IMSG_TOOL_TURN_ACK:
case FUGU_IMSG_CTX_DATA:
case FUGU_IMSG_CTX_END:
case FUGU_IMSG_TOOL_OUT:
case FUGU_IMSG_TOOL_DIFF:
case FUGU_IMSG_TOOL_RESULT:
+ case FUGU_IMSG_TOOL_ARTIFACT_ROTATE:
tool_event(coord, &imsg);
break;
+ case FUGU_IMSG_TOOL_ARTIFACT_DROP: {
+ struct imsgev_payload payload;
+
+ if (imsgev_get_payload(&imsg, &payload) == -1)
+ fatalx("bad Artifact spool retirement payload");
+ artifact_drop_event(coord, imsg_get_id(&imsg),
+ payload.data, payload.len);
+ break;
+ }
+ case FUGU_IMSG_TOOL_ARTIFACT_CANCEL_ACK: {
+ struct imsgev_payload payload;
+
+ if (imsgev_get_payload(&imsg, &payload) == -1)
+ fatalx("bad Artifact cancellation ACK payload");
+ artifact_cancel_ack_event(coord, imsg_get_id(&imsg),
+ payload.data, payload.len);
+ break;
+ }
case FUGU_IMSG_TOOL_TURN_ACK:
tool_ctl_event(coord, &imsg);
break;
}
static int
-artifact_tool_name(const char *name)
+shell_output_job_id(const struct tool_call *tc)
{
- return (strcmp(name, "shell") == 0 || strcmp(name, "grep") == 0 ||
- strcmp(name, "find") == 0 || strcmp(name, "ls") == 0 ||
- strcmp(name, "shell_bg") == 0 ||
- strcmp(name, "shell_output") == 0);
+ struct json j;
+ int root, tok, job_id = 0;
+ int64_t number;
+
+ if (json_parse(&j, tc->input.data, tc->input.len, 0) != 0)
+ return (0);
+ root = json_root(&j);
+ if (json_is_object(&j, root) &&
+ (tok = json_obj_get(&j, root, "id")) != -1 &&
+ json_get_num(&j, tok, &number) == 0 && number > 0 &&
+ number <= INT_MAX)
+ job_id = (int)number;
+ json_done(&j);
+ return (job_id);
}
static void artifact_storage_failure(struct coord *);
}
static void
+artifact_job_control(struct coord *c, uint32_t type,
+ const struct artifact_live *live)
+{
+ struct artifact_record record;
+
+ if (live->job_id <= 0)
+ return;
+ memset(&record, 0, sizeof(record));
+ strlcpy(record.id, live->stream.id, sizeof(record.id));
+ imsgev_send(&c->iev[ROLE_TOOL], type, (uint32_t)live->job_id,
+ &record, sizeof(record));
+}
+
+static int
+artifact_job_has_committed_spool(struct coord *c, int job_id)
+{
+ struct artifact_live *live;
+
+ LIST_FOREACH(live, &c->artifacts, entry)
+ if (live->job_id == job_id && live->background && live->adopted &&
+ !live->turn_owned)
+ return (1);
+ return (0);
+}
+
+static void
+artifact_background_failure(struct coord *c, struct artifact_live *failed)
+{
+ struct artifact_live *live, *next;
+ int job_id = failed->job_id;
+ int notify = 1, pending_abort;
+
+ if (job_id <= 0)
+ return;
+ LIST_FOREACH(live, &c->artifacts, entry)
+ if (live->job_id == job_id && live->job_failed)
+ notify = 0;
+ pending_abort = c->tool_ctl_expected == FUGU_IMSG_TOOL_TURN_ABORT &&
+ !artifact_job_has_committed_spool(c, job_id);
+ if (notify && !pending_abort) {
+ LIST_FOREACH(live, &c->artifacts, entry)
+ if (live->job_id == job_id)
+ live->cancel_hold = 1;
+ failed->cancel_pending = 1;
+ artifact_job_control(c, FUGU_IMSG_TOOL_ARTIFACT_CANCEL, failed);
+ }
+ LIST_FOREACH_SAFE(live, &c->artifacts, entry, next) {
+ int preserve, wake;
+
+ if (live->job_id != job_id)
+ continue;
+ live->job_failed = 1;
+ live->release_when_terminal = 0;
+ preserve = live == c->artifact_call ||
+ live == c->artifact_wait ||
+ (live == failed && c->wait == WAIT_TOOL &&
+ c->artifact_call != NULL &&
+ c->artifact_call_job_id == job_id);
+ if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE) {
+ if (!preserve)
+ artifact_live_remove(c, live);
+ continue;
+ }
+ if (!live->terminal_done) {
+ live->discard_when_terminal = 1;
+ continue;
+ }
+ wake = c->wait == WAIT_ARTIFACT_TERMINAL &&
+ c->artifact_wait == live;
+ if (!live->failed_discarded) {
+ artifact_discard(c, live);
+ live->failed_discarded = 1;
+ }
+ live->discard_when_terminal = 0;
+ if (!preserve)
+ artifact_live_remove(c, live);
+ if (wake)
+ event_loopexit(NULL);
+ }
+}
+
+/* Settle the frozen side of a published rotation when its ordinary result was
+ * abandoned. The replacement remains live for a detached committed job; a
+ * Turn-owned replacement is removed by the normal Artifact Turn abort. */
+static void
+artifact_cancelled_rotation(struct coord *c, struct artifact_live *live)
+{
+ struct artifact_live *frozen;
+
+ if (live == NULL || !live->rotation_done)
+ return;
+ frozen = artifact_rotation_frozen(c, live);
+ if (frozen == NULL) {
+ if (!live->job_failed)
+ fatalx("missing cancelled Artifact spool rotation");
+ return;
+ }
+ if (frozen->job_failed)
+ return;
+ if (frozen->stream.status == ARTIFACT_STREAM_UNAVAILABLE) {
+ artifact_live_remove(c, frozen);
+ return;
+ }
+ if (frozen->terminal_done) {
+ if (!frozen->failed_discarded)
+ artifact_discard(c, frozen);
+ if (!frozen->turn_owned && !frozen->drop_pending)
+ artifact_job_control(c,
+ FUGU_IMSG_TOOL_ARTIFACT_RELEASE, frozen);
+ artifact_live_remove(c, frozen);
+ } else {
+ frozen->discard_when_terminal = 1;
+ frozen->release_when_terminal = !frozen->turn_owned &&
+ !frozen->drop_pending;
+ }
+}
+
+static void
artifact_keep(struct coord *c, const struct artifact_live *live)
{
struct artifact_record *record;
if (live->stream.status == ARTIFACT_STREAM_AVAILABLE)
return (artifact_id_wire_valid(r->artifact_id) &&
strcmp(r->artifact_id, live->stream.id) == 0);
- return (!r->keep_artifact && artifact_id_zero(r->artifact_id) &&
- r->termination != ARTIFACT_TERM_STORAGE_ERROR);
+ return (!r->keep_artifact && artifact_id_zero(r->artifact_id));
}
static int
+artifact_cancelled_tool_result_valid(struct coord *c,
+ const struct artifact_live *live, const struct tool_result *result)
+{
+ struct artifact_live frozen;
+
+ if (live == NULL)
+ return (c->artifact_call_id == 0 &&
+ artifact_result_fields_zero(result));
+ if (c->artifact_call_id == 0 ||
+ live->correlation != c->artifact_call_id)
+ return (0);
+ if (c->artifact_call_shell_output) {
+ if (!result->artifact ||
+ result->spool_frozen != live->rotation_done)
+ return (0);
+ if (!result->spool_frozen)
+ return (!result->spool_adopted && !result->spool_detach &&
+ (result->background_job_id == 0 ||
+ result->background_job_id == live->job_id) &&
+ artifact_tool_result_valid(result, live));
+ if (!artifact_result_matches_rotation(result, live))
+ return (0);
+ memset(&frozen, 0, sizeof(frozen));
+ frozen.stream_done = 1;
+ if (artifact_id_zero(live->frozen_id))
+ frozen.stream.status = ARTIFACT_STREAM_UNAVAILABLE;
+ else {
+ frozen.stream.status = ARTIFACT_STREAM_AVAILABLE;
+ memcpy(frozen.stream.id, live->frozen_id,
+ sizeof(frozen.stream.id));
+ }
+ return (artifact_tool_result_valid(result, &frozen));
+ }
+ if (result->spool_frozen || result->spool_detach)
+ return (0);
+ if (c->artifact_call_shell_bg && result->spool_adopted) {
+ return (!result->is_error && !result->artifact &&
+ !result->keep_artifact && !result->artifact_cap_reached &&
+ result->background_job_id > 0 &&
+ result->termination == ARTIFACT_TERM_EXIT &&
+ result->observed_bytes == 0 && result->retained_bytes == 0 &&
+ artifact_result_names_call_stream(result, live));
+ }
+ return (!result->spool_adopted && result->background_job_id == 0 &&
+ artifact_tool_result_valid(result, live));
+}
+
+static int
artifact_inline_result_valid(const struct tool_result *r)
{
struct artifact_live unavailable;
static int
artifact_wait_terminal(struct coord *c, struct artifact_live *live)
{
- int rc;
+ int rc, result = 0;
while (!live->terminal_done) {
c->artifact_wait = live;
rc = event_dispatch();
if (rc == -1)
fatal("event_dispatch Artifact terminal");
- if (rc == 1 && !live->terminal_done)
- return (-1);
+ if (rc == 1 && !live->terminal_done) {
+ result = -1;
+ break;
+ }
}
c->wait = WAIT_NONE;
- c->artifact_wait = NULL;
- return (0);
+ if (c->artifact_wait == live)
+ c->artifact_wait = NULL;
+ return (result);
}
static int
-artifact_settle(struct coord *c, struct artifact_live *live,
+artifact_settle_impl(struct coord *c, struct artifact_live *live,
const struct tool_result *result)
{
struct artifact_record record;
- if (!live->stream_done ||
- !artifact_tool_result_valid(result, live))
+ if (!live->stream_done)
return (-1);
+ if (live->failed_discarded || live->job_failed) {
+ if (live->stream.status == ARTIFACT_STREAM_AVAILABLE &&
+ !live->terminal_done) {
+ live->discard_when_terminal = 1;
+ if (artifact_wait_terminal(c, live) == -1)
+ return (-1);
+ }
+ artifact_storage_failure(c);
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ return (0);
+ }
+ if (!artifact_tool_result_valid(result, live)) {
+ if (live->stream.status == ARTIFACT_STREAM_AVAILABLE) {
+ if (artifact_wait_terminal(c, live) == -1) {
+ live->discard_when_terminal = 1;
+ return (-1);
+ }
+ if (live->failed_discarded || live->job_failed) {
+ artifact_storage_failure(c);
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ return (0);
+ }
+ artifact_discard(c, live);
+ if (live->background && live->adopted && live->job_id > 0 &&
+ !live->drop_pending &&
+ live->terminal.status != ARTIFACT_STORE_ERROR)
+ artifact_job_control(c,
+ FUGU_IMSG_TOOL_ARTIFACT_RELEASE, live);
+ }
+ artifact_live_remove(c, live);
+ return (-1);
+ }
if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE) {
artifact_live_remove(c, live);
return (0);
}
if (artifact_wait_terminal(c, live) == -1)
return (-1);
- if (live->terminal.status == ARTIFACT_STORE_ERROR) {
+ if (live->failed_discarded || live->job_failed) {
artifact_storage_failure(c);
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ return (0);
+ }
+ if (live->terminal.status == ARTIFACT_STORE_ERROR ||
+ result->termination == ARTIFACT_TERM_STORAGE_ERROR) {
+ artifact_storage_failure(c);
artifact_discard(c, live);
} else {
if (live->terminal.retained_bytes != result->retained_bytes ||
(live->terminal.status == ARTIFACT_STORE_CAP &&
- !result->artifact_cap_reached))
+ !result->artifact_cap_reached)) {
+ artifact_discard(c, live);
+ if (live->background && live->adopted && live->job_id > 0 &&
+ !live->drop_pending)
+ artifact_job_control(c,
+ FUGU_IMSG_TOOL_ARTIFACT_RELEASE, live);
+ artifact_live_remove(c, live);
return (-1);
+ }
if (result->keep_artifact) {
if (live->background) {
memset(&record, 0, sizeof(record));
} else
artifact_discard(c, live);
}
+ if (live->background && live->adopted && live->job_id > 0 &&
+ !live->drop_pending &&
+ live->terminal.status != ARTIFACT_STORE_ERROR)
+ artifact_job_control(c, FUGU_IMSG_TOOL_ARTIFACT_RELEASE, live);
artifact_live_remove(c, live);
return (0);
}
+static int
+artifact_settle(struct coord *c, struct artifact_live *live,
+ const struct tool_result *result)
+{
+ int rc;
+
+ artifact_live_pin(live);
+ rc = artifact_settle_impl(c, live, result);
+ artifact_live_unpin(live);
+ return (rc);
+}
+
static void
artifact_storage_failure(struct coord *c)
{
struct artifact_read_req req;
enum artifact_page_encoding encoding;
const char *status, *message;
- int retry, rc;
+ int interrupted, retry, rc;
buf_reset(&c->tool_out);
buf_reset(&c->diff);
c->wait = WAIT_ARTIFACT_READ;
rc = event_dispatch();
c->wait = WAIT_NONE;
+ if (rc == -1)
+ fatal("event_dispatch output_read");
+ interrupted = c->cancelled || c->want_switch || c->want_quit ||
+ !c->artifact_read_done;
+ /* Cancellation can become active in the same libevent pass as exact
+ * DATA/READ_RESULT frames. Keep their correlation live and synchronously
+ * consume the bounded response before Turn rollback pumps this channel. */
+ while (!c->artifact_read_done) {
+ c->wait = WAIT_ARTIFACT_READ;
+ rc = event_dispatch();
+ c->wait = WAIT_NONE;
+ if (rc == -1)
+ fatal("event_dispatch cancelled output_read");
+ if (rc == 1 && !c->artifact_read_done)
+ fatalx("output_read lost its event source while draining");
+ }
c->artifact_read_id = 0;
memset(&c->artifact_read_req, 0, sizeof(c->artifact_read_req));
- if (rc == -1)
- fatal("event_dispatch output_read");
- if (!c->artifact_read_done) {
+ if (interrupted) {
output_read_error(c, "cancelled", "output_read was cancelled");
return;
}
{
struct imsgev *iev = &c->iev[ROLE_TOOL];
struct artifact_begin begin;
- struct artifact_live *live = NULL, *result_live = NULL;
- struct artifact_record record;
+ struct artifact_live *live = NULL, *call_live = NULL,
+ *result_live = NULL;
struct tool_req req;
uint32_t id = 0;
- int artifact, shell_bg, shell_output, rc;
+ int artifact, rotation_validated = 0, shell_bg, shell_output, rc;
- artifact = artifact_tool_name(tc->name);
+ artifact = artifact_capture_tool_name(tc->name);
shell_bg = strcmp(tc->name, "shell_bg") == 0;
shell_output = strcmp(tc->name, "shell_output") == 0;
c->artifact_call_id = 0;
+ c->artifact_call_job_id = shell_output ? shell_output_job_id(tc) : 0;
+ c->artifact_call_shell_bg = shell_bg;
+ c->artifact_call_shell_output = shell_output;
c->artifact_call = NULL;
memset(&c->tool_result, 0, sizeof(c->tool_result));
if (artifact) {
id = agent_unique_id(c);
c->artifact_call_id = id;
live = artifact_live_new(c, id, shell_bg || shell_output);
+ artifact_live_pin(live);
+ call_live = live;
c->artifact_call = live;
+ if (shell_output && c->artifact_call_job_id > 0)
+ live->job_id = c->artifact_call_job_id;
memset(&begin, 0, sizeof(begin));
begin.kind = shell_bg || shell_output ? ARTIFACT_BEGIN_SPOOL :
ARTIFACT_BEGIN_FOREGROUND;
}
if (rc == -1)
fatal("event_dispatch tool");
- if (!c->tool_done)
+ if (!c->tool_done) {
+ if (call_live != NULL) {
+ int job_id = call_live->job_id;
+
+ artifact_live_unpin(call_live);
+ if (job_id > 0)
+ artifact_cancel_retire(c, job_id);
+ }
return;
+ }
if (!artifact)
return;
+ if (!shell_output && c->tool_result.spool_frozen) {
+ c->turn_data_exceeded = 1;
+ goto done;
+ }
+ if (shell_output) {
+ if (c->tool_result.spool_frozen != live->rotation_done) {
+ c->turn_data_exceeded = 1;
+ goto done;
+ }
+ if (c->tool_result.spool_frozen) {
+ if (!artifact_result_matches_rotation(&c->tool_result,
+ live)) {
+ c->turn_data_exceeded = 1;
+ goto done;
+ }
+ rotation_validated = 1;
+ } else {
+ struct tool_result rejected;
+ const struct tool_result *settle_result = &c->tool_result;
+ int invalid;
+ invalid = c->tool_result.spool_adopted ||
+ c->tool_result.spool_detach ||
+ !c->tool_result.artifact ||
+ (c->tool_result.background_job_id != 0 &&
+ c->tool_result.background_job_id !=
+ c->artifact_call_job_id) ||
+ !artifact_result_names_call_stream(&c->tool_result,
+ live);
+ if (invalid) {
+ rejected = c->tool_result;
+ rejected.artifact = 0;
+ settle_result = &rejected;
+ }
+ if (artifact_settle(c, live, settle_result) == -1 || invalid)
+ c->turn_data_exceeded = 1;
+ live = NULL;
+ goto done;
+ }
+ }
+ if (live->job_failed) {
+ struct artifact_live *failed, *next;
+ int job_id = live->job_id;
+
+ if (artifact_settle(c, live, &c->tool_result) == -1) {
+ c->turn_data_exceeded = 1;
+ goto done;
+ }
+ live = NULL;
+ LIST_FOREACH_SAFE(failed, &c->artifacts, entry, next) {
+ if (failed->job_id != job_id || !failed->job_failed)
+ continue;
+ if (!failed->terminal_done) {
+ failed->discard_when_terminal = 1;
+ continue;
+ }
+ if (!failed->failed_discarded)
+ artifact_discard(c, failed);
+ artifact_live_remove(c, failed);
+ }
+ goto done;
+ }
if (!shell_bg && !shell_output) {
- if (c->tool_result.spool_adopted ||
+ if (c->tool_result.spool_frozen ||
+ c->tool_result.spool_adopted ||
+ c->tool_result.background_job_id != 0 ||
artifact_settle(c, live, &c->tool_result) == -1)
c->turn_data_exceeded = 1;
goto done;
}
if (shell_bg) {
+ if (c->tool_result.spool_frozen) {
+ c->turn_data_exceeded = 1;
+ goto done;
+ }
if (c->tool_result.spool_adopted) {
if (c->tool_result.artifact ||
c->tool_result.spool_detach || c->tool_result.is_error ||
+ c->tool_result.background_job_id <= 0 ||
(live->stream.status == ARTIFACT_STREAM_AVAILABLE &&
(!artifact_id_wire_valid(c->tool_result.artifact_id) ||
strcmp(c->tool_result.artifact_id,
c->turn_data_exceeded = 1;
goto done;
}
+ live->job_id = c->tool_result.background_job_id;
live->adopted = 1;
- if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE)
- artifact_live_remove(c, live);
- } else if (artifact_settle(c, live, &c->tool_result) == -1)
+ if (live->terminal_done &&
+ live->terminal.status == ARTIFACT_STORE_ERROR) {
+ artifact_background_failure(c, live);
+ artifact_storage_failure(c);
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ live = NULL;
+ }
+ } else if (c->tool_result.background_job_id != 0 ||
+ artifact_settle(c, live, &c->tool_result) == -1)
c->turn_data_exceeded = 1;
goto done;
}
c->turn_data_exceeded = 1;
goto done;
}
+ if (c->tool_result.background_job_id > 0)
+ live->job_id = c->tool_result.background_job_id;
if (artifact_id_wire_valid(c->tool_result.artifact_id))
result_live = artifact_live_by_id(c,
c->tool_result.artifact_id);
c->turn_data_exceeded = 1;
goto done;
}
- if (c->tool_result.spool_adopted) {
- live->adopted = 1;
- if (c->tool_result.spool_detach) {
- live->turn_owned = 0;
- if (live->stream.status == ARTIFACT_STREAM_AVAILABLE) {
- memset(&record, 0, sizeof(record));
- strlcpy(record.id, live->stream.id,
- sizeof(record.id));
- imsgev_send(&c->iev[ROLE_ARTIFACT],
- FUGU_IMSG_ARTIFACT_DETACH, live->correlation,
- &record, sizeof(record));
- }
- }
- } else if (c->tool_result.spool_detach) {
- c->turn_data_exceeded = 1;
- goto done;
- }
+ if ((c->tool_result.spool_adopted &&
+ c->tool_result.background_job_id <= 0) ||
+ (result_live != NULL && result_live != live &&
+ (c->tool_result.background_job_id <= 0 ||
+ result_live->job_id != c->tool_result.background_job_id)) ||
+ (c->tool_result.background_job_id == 0 && result_live != live)) {
+ int result_is_live = result_live == live;
+ if (result_live != NULL)
+ (void)artifact_settle(c, result_live, &c->tool_result);
+ if (result_is_live)
+ live = NULL;
+ c->turn_data_exceeded = 1;
+ goto done;
+ }
if (result_live != NULL) {
+ int result_is_live = result_live == live;
+ int result_failed = result_live->failed_discarded ||
+ result_live->job_failed;
+
if (artifact_settle(c, result_live, &c->tool_result) == -1)
c->turn_data_exceeded = 1;
- if (result_live == live)
+ if (result_is_live)
live = NULL;
- } else if (!artifact_inline_result_valid(&c->tool_result))
- c->turn_data_exceeded = 1;
+ else if (result_failed && live != NULL) {
+ if (live->terminal_done) {
+ if (!live->failed_discarded)
+ artifact_discard(c, live);
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ live = NULL;
+ } else
+ live->discard_when_terminal = 1;
+ }
+ } else {
+ struct artifact_live *unavailable;
+ if (rotation_validated)
+ unavailable = artifact_rotation_frozen(c, live);
+ else if (live != NULL && live->stream_done &&
+ live->stream.status == ARTIFACT_STREAM_UNAVAILABLE)
+ unavailable = live;
+ else
+ unavailable = NULL;
+ if (!artifact_inline_result_valid(&c->tool_result) ||
+ unavailable == NULL || unavailable->stream.status !=
+ ARTIFACT_STREAM_UNAVAILABLE)
+ c->turn_data_exceeded = 1;
+ if (unavailable != NULL) {
+ int result_is_live = unavailable == live;
+
+ artifact_live_remove(c, unavailable);
+ if (result_is_live)
+ live = NULL;
+ }
+ }
+
if (live != NULL && c->tool_result.spool_adopted) {
- if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE)
- artifact_live_remove(c, live);
+ if (live->terminal_done &&
+ live->terminal.status == ARTIFACT_STORE_ERROR) {
+ artifact_background_failure(c, live);
+ artifact_storage_failure(c);
+ if (!live->removed)
+ artifact_live_remove(c, live);
+ live = NULL;
+ }
} else if (live != NULL) {
if (live->stream.status == ARTIFACT_STREAM_UNAVAILABLE)
artifact_live_remove(c, live);
}
}
done:
+ if (call_live != NULL && call_live->rotation_done) {
+ struct artifact_live *frozen;
+
+ frozen = artifact_rotation_frozen(c, call_live);
+ if (frozen != NULL && !frozen->cancel_retire)
+ artifact_cancelled_rotation(c, call_live);
+ }
c->artifact_call = NULL;
c->artifact_call_id = 0;
+ c->artifact_call_job_id = 0;
+ c->artifact_call_shell_bg = 0;
+ c->artifact_call_shell_output = 0;
+ if (call_live != NULL) {
+ int job_id = call_live->job_id;
+
+ artifact_live_unpin(call_live);
+ if (job_id > 0)
+ artifact_cancel_retire(c, job_id);
+ }
}
/* Send one brokered call to fugu-web and pump events until its result. */
int role, rc;
memset(&c, 0, sizeof(c));
+ LIST_INIT(&c.artifacts);
c.conf = cf;
c.debug = debug;
c.verbose = verbose;
event_init();
/* The Artifact Role owns the lifetime Session lock. Resolve the exact
* Session id and managed root before any replay/open can mutate a Journal. */
+ artifact_root_prepare(&c);
if (journal_dir(sessdir, sizeof(sessdir)) == -1)
fatalx("cannot use ~/.fugu/sessions");
if (resume_id != NULL) {
strlcpy(c.artifact_session, id, sizeof(c.artifact_session));
c.artifact_fresh = 1;
}
- {
- const char *home = getenv("HOME");
-
- if (home == NULL || home[0] != '/' ||
- (size_t)snprintf(c.artifact_root, sizeof(c.artifact_root),
- "%s/.fugu/artifacts", home) >= sizeof(c.artifact_root))
- fatalx("cannot use ~/.fugu/artifacts");
- }
protect_snapshot_build(&c);
if ((c.devnull_fd = priv_open_devnull()) == -1)
fatal("open /dev/null");
blob - /dev/null
blob + 6f94c12c31eada82a6c98a61924ee5123dc934e7 (mode 644)
--- /dev/null
+++ src/fugu/artifact_scope.c
+/*
+ * Copyright (c) 2026 Isaac <isaac@itm.works>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ */
+
+#include <string.h>
+
+#include "artifact_scope.h"
+
+int
+artifact_capture_tool_name(const char *name)
+{
+ return (strcmp(name, "shell") == 0 || strcmp(name, "grep") == 0 ||
+ strcmp(name, "find") == 0 || strcmp(name, "ls") == 0 ||
+ strcmp(name, "shell_bg") == 0 ||
+ strcmp(name, "shell_output") == 0);
+}
blob - /dev/null
blob + f0c42eb818f25d44cc67f935b5e181aa3e802ef5 (mode 644)
--- /dev/null
+++ src/fugu/artifact_scope.h
+/*
+ * Copyright (c) 2026 Isaac <isaac@itm.works>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ */
+
+#ifndef ARTIFACT_SCOPE_H
+#define ARTIFACT_SCOPE_H
+
+/* Lead-local producers whose raw output enters managed Artifact storage. */
+int artifact_capture_tool_name(const char *);
+
+#endif /* ARTIFACT_SCOPE_H */
blob - af93fcde5229fecbd3eed0c8df674054cfb875b1
blob + bfc4f08bda7c3ccfcec4f400e53c9daa68b7f6e7
--- src/fugu/fugu.1
+++ src/fugu/fugu.1
state; a turn stopped by the 50-iteration cap is complete and remains
resumable.
Queued but undelivered messages are never journaled.
+For Lead-local shell and search tools, raw output that does not fit the bounded
+Tool result may be retained under the Owner-only managed root
+.Pa ~/.fugu/artifacts/ .
+The model receives only an opaque Session-local identifier; no storage path is
+written to the Journal or exposed through a Tool result.
+Committed retained output survives resume,
+.Cm /clear ,
+and Compaction until quota cleanup evicts it.
+Retained output can contain credentials or other secrets printed by a command;
+Owners should treat the entire managed root as sensitive data.
+At startup,
+.Nm
+may create or validate the owner-only
+.Pa ~/.fugu
+directory before resolving this root.
+If its physical path overlaps the project tree or another persistent Lead Tool
+filesystem authority,
+.Nm
+refuses the invocation before spawning workers, opening or changing a Journal,
+or contacting a Provider.
.
.Sh ENVIRONMENT
.Bl -tag -width FUGU_CONF
Personal project-independent context.
.It Pa ~/.fugu/sessions/
Per-session journals.
+.It Pa ~/.fugu/artifacts/
+Owner-only managed storage for retained Tool output; contents may include
+secrets printed by commands.
.It Pa ~/.fugu/skills/name/SKILL.md
User-installed skill.
.It Pa FUGU.md
blob - 175e290d0c0abc4258fa50bbff49452e7a488f0f
blob + f95164804fde4d69524c372dd3204fca381c4637
--- src/fugu/proto.h
+++ src/fugu/proto.h
FUGU_IMSG_TOOL_OUT, /* one result chunk (zero or more) */
FUGU_IMSG_TOOL_DIFF, /* presentation-only diff chunk (zero or more) */
FUGU_IMSG_TOOL_RESULT, /* struct tool_result: is_error; terminal */
- /* A typed pipe write end accompanies TOOL_ARTIFACT_STREAM. It is
- * accepted only by the Lead executor, never by fugu-tool -r. */
+ /* A successful shell_output rotation is durable control state, not part of
+ * the discardable result stream. It precedes TOOL_OUT/TOOL_RESULT. */
+ FUGU_IMSG_TOOL_ARTIFACT_ROTATE, /* struct artifact_spool_rotate */
+ FUGU_IMSG_TOOL_ARTIFACT_DROP, /* struct artifact_spool_drop */
+ FUGU_IMSG_TOOL_ARTIFACT_DROP_ACK, /* struct artifact_spool_drop */
+ /* A typed producer endpoint accompanies TOOL_ARTIFACT_STREAM. OpenBSD
+ * reports both pipe endpoints as O_RDWR; protocol role, not access flags,
+ * makes this the write side. Only the Lead executor accepts it. */
FUGU_IMSG_TOOL_ARTIFACT_STREAM, /* struct artifact_stream + write fd */
- FUGU_IMSG_TOOL_ARTIFACT_CANCEL, /* exact current spool id failed */
+ FUGU_IMSG_TOOL_ARTIFACT_CANCEL, /* exact job + current/historical id */
+ FUGU_IMSG_TOOL_ARTIFACT_CANCEL_ACK, /* struct artifact_job_cancel_ack */
+ FUGU_IMSG_TOOL_ARTIFACT_RELEASE, /* old spool reached settlement */
/* Private Artifact storage. Control is Coordinator-brokered; only the
* raw-byte pipe created by BEGIN forms the narrow direct data plane. */
int artifact;
int keep_artifact;
int artifact_cap_reached;
+ int spool_frozen;
int spool_adopted;
int spool_detach;
+ int background_job_id;
int termination; /* enum artifact_termination */
uint64_t observed_bytes;
uint64_t retained_bytes;
char artifact_id[FUGU_ARTIFACT_ID_BUFSZ];
};
+/* Tool -> Coordinator publication of an immutable background spool freeze.
+ * replacement_adopted says whether it was also a rotation. The imsg header
+ * id is the replacement stream's Tool-call correlation; even when no
+ * replacement was adopted it identifies the unused Turn-owned stream. A
+ * zero id array denotes the corresponding unavailable reservation. */
+struct artifact_spool_rotate {
+ uint32_t job_id;
+ uint32_t replacement_adopted;
+ uint32_t detach;
+ char frozen_id[FUGU_ARTIFACT_ID_BUFSZ];
+ char replacement_id[FUGU_ARTIFACT_ID_BUFSZ];
+};
+
+/* Tool -> Coordinator retirement of one completed, unpolled background
+ * spool, and the Coordinator's exact-key acknowledgement. The imsg header
+ * id is the Tool-call correlation that originally supplied this spool. An
+ * exactly all-zero artifact_id denotes an unavailable reservation. */
+struct artifact_spool_drop {
+ uint32_t job_id;
+ char artifact_id[FUGU_ARTIFACT_ID_BUFSZ];
+};
+
+/* Tool -> Coordinator completion of an exact job-wide cancellation. The
+ * imsg header id also equals job_id. A preceding DROP on this ordered socket
+ * was already published; an unqueued DROP for this job was removed instead. */
+struct artifact_job_cancel_ack {
+ uint32_t job_id;
+ char artifact_id[FUGU_ARTIFACT_ID_BUFSZ];
+};
+
enum artifact_stream_status {
ARTIFACT_STREAM_AVAILABLE = 0,
ARTIFACT_STREAM_UNAVAILABLE
blob - d486d07c930a48197d59feb62b0d2d5c0079ef5f
blob + 99c942ff5464de8407bbe7eb2c5268172ce75424
--- src/fugu-artifact/main.c
+++ src/fugu-artifact/main.c
*/
#include <sys/types.h>
+#include <sys/queue.h>
#include <event.h>
#include <imsg.h>
#include <stdlib.h>
#include <string.h>
#include <syslog.h>
+#include <time.h>
#include <unistd.h>
#include "buf.h"
#include "proto.h"
#include "store.h"
#include "worker.h"
+#include "xmalloc.h"
+struct drain_event;
+LIST_HEAD(drain_list, drain_event);
+
struct artifact_state {
struct worker *worker;
struct artifact_store *store;
- struct drain_event *drains;
+ struct drain_list drains;
};
struct drain_event {
- struct drain_event *next;
+ LIST_ENTRY(drain_event) entry;
struct event ev;
struct artifact_state *state;
struct artifact_pending *pending;
static struct artifact_state *cleanup_state;
static void
+artifact_store_error_test(struct artifact_terminal *terminal)
+{
+ static int injected;
+ const char *value, *errstr;
+ struct timespec delay;
+ long long ms;
+
+ if (injected || issetugid() || getenv("FUGU_SELFTEST") == NULL ||
+ (value = getenv("FUGU_ARTIFACT_STORE_ERROR_DELAY_MS")) == NULL)
+ return;
+ ms = strtonum(value, 1, 5000, &errstr);
+ if (errstr != NULL)
+ return;
+ injected = 1;
+ delay.tv_sec = (time_t)(ms / 1000);
+ delay.tv_nsec = (long)(ms % 1000) * 1000000L;
+ (void)nanosleep(&delay, NULL);
+ log_warnx("fugu-artifact: selftest STORE_ERROR injected");
+ terminal->status = ARTIFACT_STORE_ERROR;
+}
+
+static void
+artifact_read_test_delay(void)
+{
+ const char *value, *errstr;
+ struct timespec delay;
+ long long ms;
+
+ if (issetugid() ||
+ (value = getenv("FUGU_ARTIFACT_READ_DELAY_MS")) == NULL)
+ return;
+ ms = strtonum(value, 1, 5000, &errstr);
+ if (errstr != NULL)
+ return;
+ delay.tv_sec = (time_t)(ms / 1000);
+ delay.tv_nsec = (long)(ms % 1000) * 1000000L;
+ (void)nanosleep(&delay, NULL);
+}
+
+static void
drain_unlink(struct drain_event *de)
{
- struct drain_event **link;
+ struct drain_event *item;
- for (link = &de->state->drains; *link != NULL;
- link = &(*link)->next) {
- if (*link != de)
+ LIST_FOREACH(item, &de->state->drains, entry) {
+ if (item != de)
continue;
- *link = de->next;
+ LIST_REMOVE(de, entry);
return;
}
fatalx("fugu-artifact: lost drain event");
static void
drain_cancel_owned(struct artifact_state *state)
{
- struct drain_event **link, *de;
+ struct drain_event *de, *next;
- for (link = &state->drains; (de = *link) != NULL;) {
- if (!artifact_pending_turn_owned(de->pending)) {
- link = &de->next;
+ for (de = LIST_FIRST(&state->drains); de != NULL; de = next) {
+ next = LIST_NEXT(de, entry);
+ if (!artifact_pending_turn_owned(de->pending))
continue;
- }
(void)event_del(&de->ev);
- *link = de->next;
+ LIST_REMOVE(de, entry);
free(de);
}
}
rc = artifact_pending_drain(de->pending, &terminal);
if (rc == 0)
return;
+ artifact_store_error_test(&terminal);
event_del(&de->ev);
drain_unlink(de);
imsgev_send(&de->state->worker->iev, FUGU_IMSG_ARTIFACT_TERMINAL,
&stream, sizeof(stream));
return (0);
}
- de = calloc(1, sizeof(*de));
- if (de == NULL)
- fatal("calloc Artifact drain");
+ de = xcalloc(1, sizeof(*de));
de->state = state;
de->pending = pending;
de->correlation = id;
- de->next = state->drains;
- state->drains = de;
+ LIST_INSERT_HEAD(&state->drains, de, entry);
event_set(&de->ev, artifact_pending_fd(pending),
EV_READ | EV_PERSIST, drain_stream, de);
if (event_add(&de->ev, NULL) == -1)
if (id == 0 || p == NULL || len != sizeof(struct artifact_read_req))
fatalx("fugu-artifact: bad read");
+ artifact_read_test_delay();
buf_init(&out);
if (artifact_store_read(state->store,
(const struct artifact_read_req *)p, &out, &result) == -1)
-1)
fatal("fugu-artifact: pledge startup");
state.worker = &worker;
+ LIST_INIT(&state.drains);
worker.title = "fugu-artifact";
worker.ctx = &state;
worker.lockdown = artifact_lockdown;
blob - 640de1293d9cf4220161d9bcce1632da9279784c
blob + bf6bb4d01895443dae6e7d077bd2f10781b3725c
--- src/fugu-artifact/store.c
+++ src/fugu-artifact/store.c
#include <sys/types.h>
#include <sys/file.h>
+#include <sys/queue.h>
#include <sys/stat.h>
#include <dirent.h>
#include "xmalloc.h"
struct known_artifact {
- struct known_artifact *next;
+ LIST_ENTRY(known_artifact) entry;
uint64_t retained;
char id[FUGU_ARTIFACT_ID_BUFSZ];
};
struct artifact_pending {
- struct artifact_pending *next;
+ LIST_ENTRY(artifact_pending) entry;
struct artifact_store *store;
uint32_t correlation;
int file_fd;
uint64_t retained;
int terminal_done;
int terminal_status;
+ int final_linked;
int prepared;
int background;
int turn_owned;
char staging[FUGU_ARTIFACT_ID_BUFSZ + 16];
};
+LIST_HEAD(known_list, known_artifact);
+LIST_HEAD(pending_list, artifact_pending);
+
+enum artifact_store_phase {
+ ARTIFACT_STORE_RECONCILING = 0,
+ ARTIFACT_STORE_READY
+};
+
struct artifact_store {
char root[FUGU_ARTIFACT_ROOT_MAX];
char session[FUGU_ARTIFACT_SESSION_MAX];
int root_fd;
int session_lock_fd;
int global_lock_fd;
+ enum artifact_store_phase phase;
int turn_open;
- struct known_artifact *known;
- struct artifact_pending *pending;
+ int turn_prepared;
+ struct known_list known;
+ struct pending_list pending;
};
struct eviction {
if (mkdir(path, 0700) == -1 && errno != EEXIST)
return (-1);
if (lstat(path, &st) == -1 || !S_ISDIR(st.st_mode) ||
- (st.st_mode & 0777) != 0700)
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0700)
return (-1);
return (0);
}
static int
+lock_fd_normalize(int fd)
+{
+ int fdflags, flags;
+
+ if ((flags = fcntl(fd, F_GETFL)) == -1 ||
+ (flags & O_ACCMODE) != O_RDWR || (flags & O_APPEND) != 0)
+ return (-1);
+ if ((flags & O_NONBLOCK) != 0 &&
+ fcntl(fd, F_SETFL, flags & ~O_NONBLOCK) == -1)
+ return (-1);
+ if ((flags = fcntl(fd, F_GETFL)) == -1 ||
+ (flags & O_ACCMODE) != O_RDWR ||
+ (flags & (O_APPEND | O_NONBLOCK)) != 0 ||
+ (fdflags = fcntl(fd, F_GETFD)) == -1 ||
+ (fdflags & FD_CLOEXEC) == 0)
+ return (-1);
+ return (0);
+}
+
+static int
open_private(const char *path, int flags)
{
- int fd;
+ struct stat st;
+ int created = 0, fd;
- if ((fd = open(path, flags | O_CLOEXEC | O_NOFOLLOW, 0600)) == -1)
- return (-1);
- if (fchmod(fd, 0600) == -1) {
- close(fd);
- return (-1);
+ fd = open(path, (flags & ~(O_CREAT | O_EXCL)) | O_NONBLOCK |
+ O_CLOEXEC | O_NOFOLLOW);
+ if (fd == -1 && errno == ENOENT && (flags & O_CREAT)) {
+ fd = open(path, flags | O_EXCL | O_NONBLOCK | O_CLOEXEC |
+ O_NOFOLLOW, 0600);
+ if (fd != -1)
+ created = 1;
}
+ if (fd == -1)
+ return (-1);
+ if (fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) ||
+ st.st_uid != geteuid() || st.st_nlink != 1 || st.st_size != 0 ||
+ (!created && (st.st_mode & 0777) != 0600))
+ goto fail;
+ if (created && fchmod(fd, 0600) == -1)
+ goto fail;
+ if (fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) ||
+ st.st_uid != geteuid() || st.st_nlink != 1 || st.st_size != 0 ||
+ (st.st_mode & 0777) != 0600 || lock_fd_normalize(fd) == -1)
+ goto fail;
return (fd);
+fail:
+ close(fd);
+ return (-1);
}
static int
{
struct known_artifact *k;
- for (k = s->known; k != NULL; k = k->next)
+ LIST_FOREACH(k, &s->known, entry)
if (strcmp(k->id, id) == 0)
return (k);
return (NULL);
if ((k = known_find(s, id)) != NULL)
return (k->retained == retained ? 0 : -1);
k = xcalloc(1, sizeof(*k));
- strlcpy(k->id, id, sizeof(k->id));
+ memcpy(k->id, id, sizeof(k->id));
k->retained = retained;
- k->next = s->known;
- s->known = k;
+ LIST_INSERT_HEAD(&s->known, k, entry);
return (0);
}
s->root_fd = -1;
s->session_lock_fd = -1;
s->global_lock_fd = -1;
- strlcpy(s->root, root, sizeof(s->root));
- strlcpy(s->session, session, sizeof(s->session));
+ LIST_INIT(&s->known);
+ LIST_INIT(&s->pending);
+ memcpy(s->root, root, n + 1);
+ memcpy(s->session, session, strlen(session) + 1);
/* The parent ~/.fugu exists before worker startup; create only the
* dedicated managed root and this Session below it. */
if (strlcpy(fugu, root, sizeof(fugu)) >= sizeof(fugu) ||
goto fail;
if ((s->root_fd = open(root,
O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW)) == -1 ||
+ fstat(s->root_fd, &st) == -1 || !S_ISDIR(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0700 ||
(s->session_fd = open(s->session_path,
O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW)) == -1 ||
fstat(s->session_fd, &st) == -1 || !S_ISDIR(st.st_mode) ||
- (st.st_mode & 0777) != 0700 ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0700 ||
(size_t)snprintf(lock, sizeof(lock), "%s/.quota.lock", root) >=
sizeof(lock) || (s->global_lock_fd = open_private(lock,
O_RDWR | O_CREAT)) == -1)
void
artifact_store_close(struct artifact_store *s)
{
- struct known_artifact *k, *kn;
- struct artifact_pending *p, *pn;
+ struct known_artifact *k;
+ struct artifact_pending *p;
+ int dirty = 0, preserve;
if (s == NULL)
return;
- for (p = s->pending; p != NULL; p = pn) {
- pn = p->next;
+ while ((p = LIST_FIRST(&s->pending)) != NULL) {
+ LIST_REMOVE(p, entry);
if (p->read_fd != -1)
close(p->read_fd);
if (p->file_fd != -1)
close(p->file_fd);
- /* A prepared final may already be named by a synchronized Journal
- * terminal. Only replay can decide whether it is committed or an
- * orphan; clean shutdown removes staging/live spool files only. */
- if (!p->prepared)
- (void)unlinkat(s->session_fd, p->staging, 0);
- if (!p->prepared)
+ /* Only a completely prepared Turn may already be named by a
+ * synchronized Journal terminal. A partial prepare can have both
+ * names linked and is wholly rolled back on shutdown. */
+ preserve = s->turn_prepared && p->prepared;
+ if (!preserve) {
+ if (unlinkat(s->session_fd, p->staging, 0) == 0)
+ dirty = 1;
+ if (p->final_linked &&
+ unlinkat(s->session_fd, p->id, 0) == 0)
+ dirty = 1;
(void)order_unlink(s, p->id);
+ dirty = 1;
+ }
free(p);
}
- for (k = s->known; k != NULL; k = kn) {
- kn = k->next;
+ if (dirty)
+ (void)fsync(s->session_fd);
+ while ((k = LIST_FIRST(&s->known)) != NULL) {
+ LIST_REMOVE(k, entry);
free(k);
}
if (s->session_fd != -1)
artifact_store_reconcile_add(struct artifact_store *s,
const struct artifact_record *record)
{
- if (s == NULL || record == NULL || s->turn_open)
+ if (s == NULL || record == NULL ||
+ s->phase != ARTIFACT_STORE_RECONCILING || s->turn_open ||
+ !LIST_EMPTY(&s->pending))
return (-1);
return (known_add(s, record->id, record->retained_bytes));
}
return (artifact_id_valid(name));
}
+static int
+pending_name(const struct dirent *de)
+{
+ return (strncmp(de->d_name, ".pending-", 9) == 0);
+}
+
+static int
+artifact_name(const struct dirent *de)
+{
+ return (final_name(de->d_name) || pending_name(de));
+}
+
+static int
+reconcile_name(const struct dirent *de)
+{
+ return (artifact_name(de) ||
+ (strncmp(de->d_name, ORDER_PREFIX, ORDER_PREFIX_LEN) == 0 &&
+ final_name(de->d_name + ORDER_PREFIX_LEN)));
+}
+
+static int
+session_name(const struct dirent *de)
+{
+ return (de->d_name[0] != '.');
+}
+
+static void
+free_dirents(struct dirent **names, int count)
+{
+ while (count > 0)
+ free(names[--count]);
+ free(names);
+}
+
int
artifact_store_reconcile_finish(struct artifact_store *s)
{
- struct dirent *de;
- DIR *dir;
- int dfd = -1;
+ struct dirent **names = NULL;
+ const char *name, *id;
+ int count, i;
- if (s == NULL || (dfd = dup(s->session_fd)) == -1 ||
- (dir = fdopendir(dfd)) == NULL) {
- if (dfd != -1)
- close(dfd);
+ if (s == NULL || s->phase != ARTIFACT_STORE_RECONCILING ||
+ s->turn_open || !LIST_EMPTY(&s->pending) ||
+ (count = scandirat(s->session_fd, ".", &names, reconcile_name,
+ NULL)) == -1)
return (-1);
- }
- while ((de = readdir(dir)) != NULL) {
- if (strncmp(de->d_name, ".pending-", 9) == 0 ||
- (final_name(de->d_name) &&
- known_find(s, de->d_name) == NULL) ||
- (strncmp(de->d_name, ORDER_PREFIX, ORDER_PREFIX_LEN) == 0 &&
- final_name(de->d_name + ORDER_PREFIX_LEN) &&
- known_find(s, de->d_name + ORDER_PREFIX_LEN) == NULL)) {
- if (unlinkat(s->session_fd, de->d_name, 0) == -1 &&
- errno != ENOENT) {
- closedir(dir);
- return (-1);
- }
+ for (i = 0; i < count; i++) {
+ name = names[i]->d_name;
+ id = strncmp(name, ORDER_PREFIX, ORDER_PREFIX_LEN) == 0 ?
+ name + ORDER_PREFIX_LEN : name;
+ if (!pending_name(names[i]) && known_find(s, id) != NULL)
+ continue;
+ if (unlinkat(s->session_fd, name, 0) == -1 && errno != ENOENT) {
+ free_dirents(names, count);
+ return (-1);
}
}
- closedir(dir);
- return (fsync(s->session_fd));
+ free_dirents(names, count);
+ if (fault_hit(ARTIFACT_STORE_FAULT_RECONCILE_FSYNC) ||
+ fsync(s->session_fd) == -1)
+ return (-1);
+ s->phase = ARTIFACT_STORE_READY;
+ return (0);
}
static int
if (order_name(id, name, sizeof(name)) == -1)
return (-1);
- fd = openat(dirfd, name, O_RDONLY | O_CLOEXEC | O_NOFOLLOW);
+ fd = openat(dirfd, name,
+ O_RDONLY | O_NONBLOCK | O_CLOEXEC | O_NOFOLLOW);
if (fd == -1)
return (errno == ENOENT ? 1 : -1);
+ if (fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0600 ||
+ st.st_nlink != 1 || st.st_size != (off_t)sizeof(*sequence)) {
+ close(fd);
+ return (-1);
+ }
do {
n = read(fd, sequence, sizeof(*sequence));
} while (n == -1 && errno == EINTR);
- if (fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) ||
- st.st_size != (off_t)sizeof(*sequence) ||
- n != (ssize_t)sizeof(*sequence) || *sequence == 0) {
+ if (n != (ssize_t)sizeof(*sequence) || *sequence == 0) {
close(fd);
return (-1);
}
{
char path[PATH_MAX], full[PATH_MAX];
struct stat st;
- struct dirent *de;
- DIR *dir;
- int dfd, dirty = 0;
+ struct dirent **names = NULL;
+ const char *name;
+ int count, dfd, i, rc = -1;
if ((size_t)snprintf(path, sizeof(path), "%s/%s", s->root,
session) >= sizeof(path))
return (-1);
- if ((dfd = open(path, O_RDONLY | O_DIRECTORY | O_CLOEXEC |
- O_NOFOLLOW)) == -1)
+ if (current)
+ dfd = dup(s->session_fd);
+ else
+ dfd = openat(s->root_fd, session,
+ O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW);
+ if (dfd == -1)
return (errno == ENOENT ? 0 : -1);
- if ((dir = fdopendir(dfd)) == NULL) {
+ if (fstat(dfd, &st) == -1 || !S_ISDIR(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0700) {
close(dfd);
return (-1);
}
- while ((de = readdir(dir)) != NULL) {
- if (!final_name(de->d_name) &&
- strncmp(de->d_name, ".pending-", 9) != 0)
- continue;
- /* Once its lifetime lock is held, another Session cannot have a
- * live producer. Staging entries there are crash orphans, not quota
- * reservations; remove and synchronize them before accounting. */
- if (!current && evictable &&
- strncmp(de->d_name, ".pending-", 9) == 0) {
+ /* A lifetime-lock-proven inactive Session has no live producer. Remove
+ * all crash staging names as one durable pass before validating or
+ * counting finals, because a crash after linkat can leave the two names
+ * temporarily sharing an otherwise valid inode. */
+ if (!current && evictable) {
+ count = scandirat(dfd, ".", &names, pending_name, NULL);
+ if (count == -1)
+ goto done;
+ for (i = 0; i < count; i++) {
char order[FUGU_ARTIFACT_ID_BUFSZ + ORDER_PREFIX_LEN];
+ const char *id;
+ int have_final = 0;
- if (unlinkat(dirfd(dir), de->d_name, 0) == -1 &&
- errno != ENOENT) {
- closedir(dir);
- return (-1);
+ name = names[i]->d_name;
+ id = name + 9;
+ if (final_name(id)) {
+ if (fstatat(dfd, id, &st,
+ AT_SYMLINK_NOFOLLOW) == 0)
+ have_final = 1;
+ else if (errno != ENOENT)
+ goto done;
}
- if (final_name(de->d_name + 9) &&
- order_name(de->d_name + 9, order, sizeof(order)) == 0 &&
- unlinkat(dirfd(dir), order, 0) == -1 && errno != ENOENT) {
- closedir(dir);
- return (-1);
- }
- dirty = 1;
- continue;
+ if (unlinkat(dfd, name, 0) == -1 && errno != ENOENT)
+ goto done;
+ /* The order record belongs to an already-linked final when one
+ * exists; pending-only crash state owns its order record. */
+ if (!have_final && final_name(id) &&
+ order_name(id, order, sizeof(order)) == 0 &&
+ unlinkat(dfd, order, 0) == -1 && errno != ENOENT)
+ goto done;
}
- if ((size_t)snprintf(full, sizeof(full), "%s/%s", path,
- de->d_name) >= sizeof(full) || lstat(full, &st) == -1 ||
- !S_ISREG(st.st_mode) || st.st_size < 0) {
- closedir(dir);
- return (-1);
- }
- if ((uint64_t)st.st_size > UINT64_MAX - *global_total) {
- closedir(dir);
- return (-1);
- }
+ free_dirents(names, count);
+ names = NULL;
+ if (fault_hit(ARTIFACT_STORE_FAULT_INACTIVE_CLEANUP_FSYNC) ||
+ fsync(dfd) == -1)
+ goto done;
+ }
+ count = scandirat(dfd, ".", &names, artifact_name, NULL);
+ if (count == -1)
+ goto done;
+ for (i = 0; i < count; i++) {
+ int is_final;
+
+ name = names[i]->d_name;
+ is_final = final_name(name);
+ if (fstatat(dfd, name, &st,
+ AT_SYMLINK_NOFOLLOW) == -1 || !S_ISREG(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0600 ||
+ st.st_nlink != 1 || st.st_size < 0 ||
+ (uint64_t)st.st_size > ARTIFACT_RAW_MAX)
+ goto done;
+ if ((uint64_t)st.st_size > UINT64_MAX - *global_total)
+ goto done;
*global_total += (uint64_t)st.st_size;
- if (current)
+ if (current) {
+ if ((uint64_t)st.st_size > UINT64_MAX - *session_total)
+ goto done;
*session_total += (uint64_t)st.st_size;
- if (!evictable || !final_name(de->d_name))
+ }
+ if (!evictable || !is_final)
continue;
+ if ((size_t)snprintf(full, sizeof(full), "%s/%s", path,
+ name) >= sizeof(full))
+ goto done;
if (*nlist == *cap) {
*cap = *cap == 0 ? 16 : *cap * 2;
*list = xreallocarray(*list, *cap, sizeof(**list));
}
memset(&(*list)[*nlist], 0, sizeof(**list));
- strlcpy((*list)[*nlist].path, full,
- sizeof((*list)[*nlist].path));
+ memcpy((*list)[*nlist].path, full, strlen(full) + 1);
(*list)[*nlist].size = st.st_size;
(*list)[*nlist].mtime = st.st_mtim;
{
- int rc = read_order(dirfd(dir), de->d_name,
+ int order_rc = read_order(dfd, name,
&(*list)[*nlist].sequence);
- if (rc == -1) {
- closedir(dir);
- return (-1);
- }
- (*list)[*nlist].have_sequence = rc == 0;
+ if (order_rc == -1)
+ goto done;
+ (*list)[*nlist].have_sequence = order_rc == 0;
}
if ((size_t)snprintf((*list)[*nlist].order_path,
sizeof((*list)[*nlist].order_path), "%s/%s%s", path,
- ORDER_PREFIX, de->d_name) >=
- sizeof((*list)[*nlist].order_path)) {
- closedir(dir);
- return (-1);
- }
- strlcpy((*list)[*nlist].dir_path, path,
- sizeof((*list)[*nlist].dir_path));
+ ORDER_PREFIX, name) >=
+ sizeof((*list)[*nlist].order_path))
+ goto done;
+ memcpy((*list)[*nlist].dir_path, path, strlen(path) + 1);
(*list)[*nlist].current = current;
(*nlist)++;
}
- if (dirty && fsync(dirfd(dir)) == -1) {
- closedir(dir);
- return (-1);
- }
- closedir(dir);
- return (0);
+ rc = 0;
+done:
+ if (names != NULL)
+ free_dirents(names, count);
+ close(dfd);
+ return (rc);
}
static int
static int
other_session_lock(struct artifact_store *s, const char *session)
{
- char path[PATH_MAX];
+ struct stat st;
int dfd, fd;
- if ((size_t)snprintf(path, sizeof(path), "%s/%s", s->root,
- session) >= sizeof(path) ||
- (dfd = open(path, O_RDONLY | O_DIRECTORY | O_CLOEXEC |
- O_NOFOLLOW)) == -1)
+ if (!session_valid(session) || (dfd = openat(s->root_fd, session,
+ O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW)) == -1)
return (-1);
- fd = openat(dfd, ".session.lock", O_RDWR | O_CLOEXEC | O_NOFOLLOW);
+ if (fstat(dfd, &st) == -1 || !S_ISDIR(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0700) {
+ close(dfd);
+ return (-1);
+ }
+ fd = openat(dfd, ".session.lock",
+ O_RDWR | O_NONBLOCK | O_CLOEXEC | O_NOFOLLOW);
close(dfd);
if (fd == -1)
return (-1);
+ if (fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0600 ||
+ st.st_nlink != 1 || st.st_size != 0 ||
+ lock_fd_normalize(fd) == -1) {
+ close(fd);
+ return (-1);
+ }
if (flock(fd, LOCK_EX | LOCK_NB) == -1) {
close(fd);
return (-1);
reserve_space(struct artifact_store *s)
{
struct eviction *list = NULL;
- struct dirent *de;
- DIR *root;
+ struct dirent **names = NULL;
int *locks = NULL;
uint64_t session_total = 0, global_total = 0;
size_t nlist = 0, cap = 0, i, nlocks = 0;
- int dfd = -1, lockfd, ok = -1, scan_ok = 0;
+ int count = 0, j, lockfd, ok = -1, scan_ok = 0;
if (flock(s->global_lock_fd, LOCK_EX) == -1)
return (-1);
if (scan_session(s, s->session, 1, 1, &session_total,
&global_total, &list, &nlist, &cap) == -1)
goto done;
- if ((dfd = open(s->root, O_RDONLY | O_DIRECTORY | O_CLOEXEC |
- O_NOFOLLOW)) == -1 || (root = fdopendir(dfd)) == NULL) {
- if (dfd != -1)
- close(dfd);
+ if ((count = scandirat(s->root_fd, ".", &names, session_name,
+ NULL)) == -1)
goto done;
- }
- while ((de = readdir(root)) != NULL) {
- if (strcmp(de->d_name, ".") == 0 ||
- strcmp(de->d_name, "..") == 0 ||
- strcmp(de->d_name, s->session) == 0 ||
- de->d_name[0] == '.')
+ for (j = 0; j < count; j++) {
+ const char *name = names[j]->d_name;
+
+ if (strcmp(name, s->session) == 0)
continue;
- lockfd = other_session_lock(s, de->d_name);
+ lockfd = other_session_lock(s, name);
if (lockfd != -1) {
locks = xreallocarray(locks, nlocks + 1, sizeof(*locks));
locks[nlocks++] = lockfd;
}
- if (scan_session(s, de->d_name, 0, lockfd != -1, &session_total,
+ if (scan_session(s, name, 0, lockfd != -1, &session_total,
&global_total, &list, &nlist, &cap) == -1) {
- closedir(root);
goto done;
}
}
- closedir(root);
+ free_dirents(names, count);
+ names = NULL;
qsort(list, nlist, sizeof(*list), eviction_cmp);
/* Satisfy the per-Session bound from that Session's own oldest finals. */
for (i = 0; session_total >
* or quota lock is released, including partial-error paths. */
if (eviction_sync(list, nlist) == -1)
ok = -1;
+ if (names != NULL)
+ free_dirents(names, count);
free(list);
for (i = 0; i < nlocks; i++) {
(void)flock(locks[i], LOCK_UN);
{
struct artifact_pending *p;
- if (s == NULL || s->turn_open)
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY || s->turn_open ||
+ s->turn_prepared)
return (-1);
- for (p = s->pending; p != NULL; p = p->next)
+ LIST_FOREACH(p, &s->pending, entry)
if (p->turn_owned)
return (-1);
s->turn_open = 1;
int pipes[2], tries;
if (s == NULL || out == NULL || write_fd == NULL || stream == NULL ||
- !s->turn_open || correlation == 0 ||
+ s->phase != ARTIFACT_STORE_READY || !s->turn_open ||
+ s->turn_prepared ||
+ correlation == 0 ||
pending_find_correlation(s, correlation) != NULL ||
(kind != ARTIFACT_BEGIN_FOREGROUND && kind != ARTIFACT_BEGIN_SPOOL))
return (-1);
continue;
if (errno != ENOENT)
break;
- (void)snprintf(p->staging, sizeof(p->staging), ".pending-%s",
- p->id);
+ if ((size_t)snprintf(p->staging, sizeof(p->staging),
+ ".pending-%s", p->id) >= sizeof(p->staging))
+ break;
p->file_fd = openat(s->session_fd, p->staging,
O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW | O_CLOEXEC, 0600);
if (p->file_fd != -1 || errno != EEXIST)
}
(void)flock(s->global_lock_fd, LOCK_UN);
p->read_fd = pipes[0];
- p->next = s->pending;
- s->pending = p;
+ LIST_INSERT_HEAD(&s->pending, p, entry);
stream->status = ARTIFACT_STREAM_AVAILABLE;
- strlcpy(stream->id, p->id, sizeof(stream->id));
+ memcpy(stream->id, p->id, sizeof(stream->id));
*out = p;
*write_fd = pipes[1];
return (0);
struct stat st;
uint64_t sequence = 0;
ssize_t n;
- int counter = -1, order = -1, rc = -1;
+ int counter = -1, counter_created = 0, order = -1, rc = -1;
if (fault_hit(ARTIFACT_STORE_FAULT_DRAIN_ORDER) ||
order_name(p->id, name, sizeof(name)) == -1 ||
flock(s->global_lock_fd, LOCK_EX) == -1)
return (-1);
counter = openat(s->root_fd, SEQUENCE_FILE,
- O_RDWR | O_CREAT | O_CLOEXEC | O_NOFOLLOW, 0600);
- if (counter == -1 || fchmod(counter, 0600) == -1 ||
- fstat(counter, &st) == -1 || !S_ISREG(st.st_mode) ||
+ O_RDWR | O_NONBLOCK | O_CLOEXEC | O_NOFOLLOW);
+ if (counter == -1 && errno == ENOENT) {
+ counter = openat(s->root_fd, SEQUENCE_FILE,
+ O_RDWR | O_NONBLOCK | O_CREAT | O_EXCL | O_CLOEXEC |
+ O_NOFOLLOW, 0600);
+ if (counter != -1)
+ counter_created = 1;
+ }
+ /* Existing metadata is authenticated before any chmod or write. A new
+ * O_EXCL inode may be repaired after an inherited umask, then undergoes
+ * the same complete authentication before its first write. */
+ if (counter == -1 || fstat(counter, &st) == -1 ||
+ !S_ISREG(st.st_mode) || st.st_uid != geteuid() ||
+ st.st_nlink != 1 ||
+ (!counter_created && (st.st_mode & 0777) != 0600) ||
(st.st_size != 0 && st.st_size != (off_t)sizeof(sequence)))
goto done;
+ if (counter_created && fchmod(counter, 0600) == -1)
+ goto done;
+ if (fstat(counter, &st) == -1 || !S_ISREG(st.st_mode) ||
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0600 ||
+ st.st_nlink != 1 ||
+ (st.st_size != 0 && st.st_size != (off_t)sizeof(sequence)))
+ goto done;
if (st.st_size != 0) {
do {
n = pread(counter, &sequence, sizeof(sequence), 0);
ftruncate(counter, sizeof(sequence)) == -1 || fsync(counter) == -1)
goto done;
order = openat(s->session_fd, name,
- O_WRONLY | O_CREAT | O_EXCL | O_CLOEXEC | O_NOFOLLOW, 0600);
- if (order == -1 || fchmod(order, 0600) == -1)
+ O_WRONLY | O_NONBLOCK | O_CREAT | O_EXCL | O_CLOEXEC |
+ O_NOFOLLOW, 0600);
+ if (order == -1 || fchmod(order, 0600) == -1 ||
+ fstat(order, &st) == -1 ||
+ !S_ISREG(st.st_mode) || st.st_uid != geteuid() ||
+ (st.st_mode & 0777) != 0600 || st.st_nlink != 1 || st.st_size != 0)
goto done;
do {
n = write(order, &sequence, sizeof(sequence));
p->terminal_done = 1;
p->terminal_status = terminal->status;
terminal->retained_bytes = p->retained;
- strlcpy(terminal->id, p->id, sizeof(terminal->id));
+ memcpy(terminal->id, p->id, sizeof(terminal->id));
return (result);
}
static struct artifact_pending *
-pending_find(struct artifact_store *s, const char *id,
- struct artifact_pending ***linkp)
+pending_find(struct artifact_store *s, const char *id)
{
- struct artifact_pending **link, *p;
+ struct artifact_pending *p;
- for (link = &s->pending; (p = *link) != NULL; link = &p->next) {
- if (strcmp(p->id, id) == 0) {
- if (linkp != NULL)
- *linkp = link;
+ LIST_FOREACH(p, &s->pending, entry)
+ if (strcmp(p->id, id) == 0)
return (p);
- }
- }
return (NULL);
}
{
struct artifact_pending *p;
- for (p = s->pending; p != NULL; p = p->next)
+ LIST_FOREACH(p, &s->pending, entry)
if (p->correlation == correlation)
return (p);
return (NULL);
static struct artifact_pending *
pending_find_pair(struct artifact_store *s, uint32_t correlation,
- const char *id, struct artifact_pending ***linkp)
+ const char *id)
{
struct artifact_pending *p;
if (correlation == 0 ||
- (p = pending_find(s, id, linkp)) == NULL ||
+ (p = pending_find(s, id)) == NULL ||
p->correlation != correlation)
return (NULL);
return (p);
artifact_store_discard(struct artifact_store *s, uint32_t correlation,
const char *id)
{
- struct artifact_pending **link, *p;
+ struct artifact_pending *p;
- if (s == NULL || !artifact_id_wire_valid(id) ||
- (p = pending_find_pair(s, correlation, id, &link)) == NULL ||
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY ||
+ !artifact_id_wire_valid(id) ||
+ (p = pending_find_pair(s, correlation, id)) == NULL ||
!p->terminal_done ||
- p->prepared)
+ p->final_linked || p->prepared)
return (-1);
if (p->file_fd != -1) {
close(p->file_fd);
return (-1);
if (order_unlink(s, p->id) == -1)
return (-1);
- *link = p->next;
+ LIST_REMOVE(p, entry);
free(p);
return (0);
}
{
struct artifact_pending *p;
- if (s == NULL || !s->turn_open || !artifact_id_wire_valid(id) ||
- (p = pending_find_pair(s, correlation, id, NULL)) == NULL ||
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY || !s->turn_open ||
+ !artifact_id_wire_valid(id) ||
+ (p = pending_find_pair(s, correlation, id)) == NULL ||
!p->terminal_done ||
(p->terminal_status != ARTIFACT_STORE_OK &&
p->terminal_status != ARTIFACT_STORE_CAP) || !p->background ||
- p->prepared)
+ p->final_linked || p->prepared)
return (-1);
p->background = 0;
p->turn_owned = 1;
{
struct artifact_pending *p;
- if (s == NULL || !s->turn_open || !artifact_id_wire_valid(id) ||
- (p = pending_find_pair(s, correlation, id, NULL)) == NULL ||
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY || !s->turn_open ||
+ !artifact_id_wire_valid(id) ||
+ (p = pending_find_pair(s, correlation, id)) == NULL ||
!p->background ||
- !p->turn_owned || p->prepared)
+ !p->turn_owned || p->final_linked || p->prepared)
return (-1);
p->turn_owned = 0;
return (0);
{
struct artifact_pending *p;
- if (s == NULL || !s->turn_open)
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY || !s->turn_open ||
+ s->turn_prepared)
return (-1);
- for (p = s->pending; p != NULL; p = p->next) {
+ LIST_FOREACH(p, &s->pending, entry) {
if (p->terminal_done &&
p->terminal_status == ARTIFACT_STORE_ERROR)
return (-1);
p->terminal_status != ARTIFACT_STORE_CAP) || p->prepared)
return (-1);
}
- for (p = s->pending; p != NULL; p = p->next) {
+ LIST_FOREACH(p, &s->pending, entry) {
if (p->background)
continue;
/* linkat is the no-replace publication primitive: renameat would
* silently overwrite a final and let an expired id alias new bytes. */
if (fsync(p->file_fd) == -1 ||
linkat(s->session_fd, p->staging, s->session_fd, p->id, 0) ==
- -1 || unlinkat(s->session_fd, p->staging, 0) == -1)
+ -1)
return (-1);
+ p->final_linked = 1;
+ if (fault_hit(ARTIFACT_STORE_FAULT_PREPARE_STAGING_UNLINK) ||
+ unlinkat(s->session_fd, p->staging, 0) == -1)
+ return (-1);
p->prepared = 1;
}
- return (fsync(s->session_fd));
+ if (fsync(s->session_fd) == -1)
+ return (-1);
+ s->turn_prepared = 1;
+ return (0);
}
int
artifact_store_turn_commit(struct artifact_store *s)
{
- struct artifact_pending **link, *p;
+ struct artifact_pending *p, *next;
- if (s == NULL || !s->turn_open)
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY || !s->turn_open ||
+ !s->turn_prepared)
return (-1);
- for (link = &s->pending; (p = *link) != NULL;) {
+ for (p = LIST_FIRST(&s->pending); p != NULL; p = next) {
+ next = LIST_NEXT(p, entry);
if (p->background) {
p->turn_owned = 0;
- link = &p->next;
continue;
}
if (!p->prepared || known_add(s, p->id, p->retained) == -1)
return (-1);
close(p->file_fd);
- *link = p->next;
+ LIST_REMOVE(p, entry);
free(p);
}
s->turn_open = 0;
+ s->turn_prepared = 0;
return (0);
}
int
artifact_store_turn_abort(struct artifact_store *s)
{
- struct artifact_pending **link, *p;
+ struct artifact_pending *p, *next;
int failed = 0, dirty = 0;
- if (s == NULL || !s->turn_open)
+ if (s == NULL || s->phase != ARTIFACT_STORE_READY || !s->turn_open)
return (-1);
- for (p = s->pending; p != NULL; p = p->next) {
+ s->turn_prepared = 0;
+ LIST_FOREACH(p, &s->pending, entry) {
if (!p->turn_owned)
continue;
if (p->read_fd != -1) {
close(p->file_fd);
p->file_fd = -1;
}
- if (fault_hit(p->prepared ?
- ARTIFACT_STORE_FAULT_ABORT_FINAL_UNLINK :
- ARTIFACT_STORE_FAULT_ABORT_STAGING_UNLINK) ||
- (unlinkat(s->session_fd,
- p->prepared ? p->id : p->staging, 0) == -1 &&
+ if (fault_hit(ARTIFACT_STORE_FAULT_ABORT_STAGING_UNLINK) ||
+ (unlinkat(s->session_fd, p->staging, 0) == -1 &&
errno != ENOENT))
failed = 1;
else
dirty = 1;
+ if (p->final_linked &&
+ (fault_hit(ARTIFACT_STORE_FAULT_ABORT_FINAL_UNLINK) ||
+ (unlinkat(s->session_fd, p->id, 0) == -1 &&
+ errno != ENOENT)))
+ failed = 1;
+ else if (p->final_linked)
+ dirty = 1;
if (fault_hit(ARTIFACT_STORE_FAULT_ABORT_ORDER_UNLINK) ||
order_unlink(s, p->id) == -1)
failed = 1;
failed = 1;
if (failed)
return (-1);
- for (link = &s->pending; (p = *link) != NULL;) {
- if (!p->turn_owned) {
- link = &p->next;
+ for (p = LIST_FIRST(&s->pending); p != NULL; p = next) {
+ next = LIST_NEXT(p, entry);
+ if (!p->turn_owned)
continue;
- }
- *link = p->next;
+ LIST_REMOVE(p, entry);
free(p);
}
s->turn_open = 0;
int fd = -1;
int pending = 0;
+ if (s == NULL || out == NULL || result == NULL ||
+ s->phase != ARTIFACT_STORE_READY)
+ return (-1);
buf_reset(out);
memset(result, 0, sizeof(*result));
- if (s == NULL || req == NULL || !artifact_id_wire_valid(req->id)) {
+ if (req == NULL || !artifact_id_wire_valid(req->id)) {
result->status = ARTIFACT_READ_MALFORMED;
return (0);
}
result->status = ARTIFACT_READ_RANGE;
return (0);
}
- if ((p = pending_find(s, req->id, NULL)) != NULL) {
+ if ((p = pending_find(s, req->id)) != NULL) {
fd = p->file_fd;
result->total_bytes = p->retained;
result->status = ARTIFACT_READ_PENDING;
} else if ((known = known_find(s, req->id)) != NULL) {
result->total_bytes = known->retained;
fd = openat(s->session_fd, req->id,
- O_RDONLY | O_CLOEXEC | O_NOFOLLOW);
+ O_RDONLY | O_NONBLOCK | O_CLOEXEC | O_NOFOLLOW);
if (fd == -1 && errno == ENOENT) {
result->status = ARTIFACT_READ_EXPIRED;
return (0);
return (0);
}
if (fd == -1 || fstat(fd, &st) == -1 || !S_ISREG(st.st_mode) ||
- st.st_size < 0 || (pending ?
+ st.st_uid != geteuid() || (st.st_mode & 0777) != 0600 ||
+ st.st_nlink != 1 || st.st_size < 0 || (pending ?
(uint64_t)st.st_size < result->total_bytes :
(uint64_t)st.st_size != result->total_bytes)) {
result->status = ARTIFACT_READ_ERROR;
blob - e0eb536cbc2b88f91b61d63beeed573ae12392ab
blob + 53ad88a2dd0728d10fef8858ecadf4dcace41e3d
--- src/fugu-artifact/store.h
+++ src/fugu-artifact/store.h
ARTIFACT_STORE_FAULT_DRAIN_FTRUNCATE,
ARTIFACT_STORE_FAULT_DRAIN_FUTIMENS,
ARTIFACT_STORE_FAULT_DRAIN_ORDER,
+ ARTIFACT_STORE_FAULT_RECONCILE_FSYNC,
+ ARTIFACT_STORE_FAULT_PREPARE_STAGING_UNLINK,
ARTIFACT_STORE_FAULT_ABORT_STAGING_UNLINK,
ARTIFACT_STORE_FAULT_ABORT_FINAL_UNLINK,
ARTIFACT_STORE_FAULT_ABORT_ORDER_UNLINK,
ARTIFACT_STORE_FAULT_ABORT_FSYNC,
ARTIFACT_STORE_FAULT_DISCARD_UNLINK,
+ ARTIFACT_STORE_FAULT_INACTIVE_CLEANUP_FSYNC,
ARTIFACT_STORE_FAULT_EVICTION_FSYNC
};
blob - 25d5a1715dc1a828482e9a376c84e3e696389500
blob + d20aac06cfa37b15618a2fb9c3355db0abf7cb9f
--- src/fugu-tool/Makefile
+++ src/fugu-tool/Makefile
anthropic_req.c claude_sub_req.c claude_sub_profile.c openai_req.c \
codex_sub_req.c codex_sub_profile.c \
turn_mechanics.c generation.c msg.c provider_state.c json.c \
- artifact_result.c artifact_capture.c effort.c buf.c diff.c log.c \
- xmalloc.c
+ artifact_capture.c artifact_pipe.c artifact_scope.c \
+ artifact_result.c effort.c buf.c diff.c log.c xmalloc.c
BINDIR= ${LIBEXECDIR}/fugu
MAN=
blob - 94d6edd0df8f48460e37801d9c44410c5fe00091
blob + ba537a15eb22ee38f21810488368ceb153bbbf93
--- src/fugu-tool/job.c
+++ src/fugu-tool/job.c
*/
#include <sys/types.h>
+#include <sys/ioctl.h>
+#include <sys/queue.h>
#include <sys/wait.h>
#include <errno.h>
#include <fcntl.h>
#include <limits.h>
+#include <poll.h>
#include <signal.h>
#include <stdint.h>
#include <stdio.h>
#define JOB_PUMP_TOTAL (1024 * 1024)
#define JOB_WRITE_PENDING_MAX (64 * 1024)
#define DEFAULT_CAP 4 /* max_bg_jobs default */
+#define JOB_COMPLETED_RECORD_MAX 64 /* separate from running-job cap */
+#define JOB_TOMBSTONE_MAX 512 /* exact delayed-control keys */
+#define JOB_OWNERSHIP_MAX 1024 /* all unresolved job/spool keys */
+struct job_artifact_ref {
+ LIST_ENTRY(job_artifact_ref) entry;
+ char id[ARTIFACT_ID_HEX_LEN + 1];
+};
+LIST_HEAD(job_artifact_ref_list, job_artifact_ref);
+
+struct job_artifact_tombstone {
+ LIST_ENTRY(job_artifact_tombstone) entry;
+ int job_id;
+ uint32_t correlation;
+ int drop_pending;
+ int drop_sent;
+ char id[ARTIFACT_ID_HEX_LEN + 1];
+};
+LIST_HEAD(job_artifact_tombstone_list, job_artifact_tombstone);
+
struct job {
struct job *next;
int id;
int artifact_fd; /* current spool write end, nonblocking */
int artifact_managed;
int artifact_available;
+ int artifact_settling;
+ uint32_t artifact_correlation;
char artifact_id[ARTIFACT_ID_HEX_LEN + 1];
+ struct job_artifact_ref_list artifact_history;
struct artifact_capture capture; /* bounded current segment preview */
struct buf pending_write; /* bounded partial spool write */
int cap_reached;
int exited;
int status;
int pending; /* spawned by the open, uncommitted Turn */
+ int reported_terminal;
+#ifdef JOB_TESTING
+ size_t test_pump_bytes;
+#endif
};
struct joblist {
struct job *head;
+ struct job_artifact_tombstone_list tombstones;
int64_t next_id;
int cap;
+ size_t record_limit;
+ size_t tombstone_count;
+ size_t tombstone_limit;
+ size_t ownership_limit;
int turn_open;
int pump_cursor; /* last id visited by round-robin pump */
+#ifdef JOB_TESTING
+ size_t test_pump_total;
+#endif
};
static void job_stop(struct job *);
+static struct job *job_find(struct joblist *, int);
+static void job_prune_reported(struct joblist *);
+static size_t job_ownership_count(struct joblist *);
+#ifdef JOB_TESTING
+static int test_backpressure_at_boundary;
+static size_t test_pump_per_job = JOB_PUMP_PER_JOB;
+static size_t test_pump_total = JOB_PUMP_TOTAL;
+
+void
+job_test_backpressure_at_boundary(void)
+{
+ test_backpressure_at_boundary = 1;
+}
+
+int
+job_test_boundary_pending(struct joblist *jl, int id)
+{
+ struct job *j = job_find(jl, id);
+
+ return (j != NULL && j->capture.retained == ARTIFACT_RAW_MAX &&
+ j->pending_write.len > 0);
+}
+
+int
+job_test_storage_unavailable(struct joblist *jl, int id)
+{
+ struct job *j = job_find(jl, id);
+
+ return (j != NULL && j->storage_unavailable);
+}
+
+size_t
+job_test_pending_bytes(struct joblist *jl, int id)
+{
+ struct job *j = job_find(jl, id);
+
+ return (j != NULL ? j->pending_write.len : 0);
+}
+
+size_t
+job_test_output_bytes(struct joblist *jl, int id)
+{
+ struct job *j = job_find(jl, id);
+ int bytes = 0;
+
+ if (j == NULL || j->fd == -1 || ioctl(j->fd, FIONREAD, &bytes) == -1 ||
+ bytes < 0)
+ return (0);
+ return ((size_t)bytes);
+}
+
+void
+job_test_set_pump_limits(size_t per_job, size_t total)
+{
+ test_pump_per_job = per_job;
+ test_pump_total = total;
+}
+
+void
+job_test_reset_pump_limits(void)
+{
+ test_pump_per_job = JOB_PUMP_PER_JOB;
+ test_pump_total = JOB_PUMP_TOTAL;
+}
+
+size_t
+job_test_pump_per_job_limit(void)
+{
+ return (JOB_PUMP_PER_JOB);
+}
+
+size_t
+job_test_pump_total_limit(void)
+{
+ return (JOB_PUMP_TOTAL);
+}
+
+size_t
+job_test_last_pump_total(struct joblist *jl)
+{
+ return (jl->test_pump_total);
+}
+
+size_t
+job_test_last_pump_bytes(struct joblist *jl, int id)
+{
+ struct job *j = job_find(jl, id);
+
+ return (j != NULL ? j->test_pump_bytes : 0);
+}
+
+int
+job_test_state(struct joblist *jl, int id, struct job_test_state *state)
+{
+ struct job *j = job_find(jl, id);
+
+ if (j == NULL || state == NULL)
+ return (-1);
+ memset(state, 0, sizeof(*state));
+ state->observed = j->capture.observed;
+ state->retained = j->capture.retained;
+ state->pending_bytes = j->pending_write.len;
+ state->cap_reached = j->cap_reached;
+ state->exited = j->exited;
+ state->producer_open = j->fd != -1;
+ state->artifact_open = j->artifact_fd != -1;
+ state->storage_error = j->storage_error;
+ state->storage_unavailable = j->storage_unavailable;
+ return (0);
+}
+
+void
+job_test_set_record_limit(struct joblist *jl, size_t limit)
+{
+ jl->record_limit = limit;
+}
+
+void
+job_test_set_tombstone_limit(struct joblist *jl, size_t limit)
+{
+ jl->tombstone_limit = limit;
+}
+
+size_t
+job_test_record_count(struct joblist *jl)
+{
+ struct job *j;
+ size_t n = 0;
+
+ for (j = jl->head; j != NULL; j = j->next)
+ n++;
+ return (n);
+}
+
+size_t
+job_test_tombstone_count(struct joblist *jl)
+{
+ return (jl->tombstone_count);
+}
+
+void
+job_test_set_ownership_limit(struct joblist *jl, size_t limit)
+{
+ jl->ownership_limit = limit;
+}
+
+size_t
+job_test_ownership_count(struct joblist *jl)
+{
+ return (job_ownership_count(jl));
+}
+#endif
+
+static void
+job_history_free(struct job *j)
+{
+ struct job_artifact_ref *ref;
+
+ while ((ref = LIST_FIRST(&j->artifact_history)) != NULL) {
+ LIST_REMOVE(ref, entry);
+ free(ref);
+ }
+}
+
+static int
+job_history_has(const struct job *j, const char *id)
+{
+ const struct job_artifact_ref *ref;
+
+ LIST_FOREACH(ref, &j->artifact_history, entry)
+ if (strcmp(ref->id, id) == 0)
+ return (1);
+ return (0);
+}
+
+static int
+job_history_add(struct job *j, const char *id)
+{
+ struct job_artifact_ref *ref;
+
+ if (!artifact_id_wire_valid(id) || job_history_has(j, id))
+ return (-1);
+ ref = xcalloc(1, sizeof(*ref));
+ strlcpy(ref->id, id, sizeof(ref->id));
+ LIST_INSERT_HEAD(&j->artifact_history, ref, entry);
+ return (0);
+}
+
+static int
+job_history_remove(struct job *j, const char *id)
+{
+ struct job_artifact_ref *ref;
+
+ LIST_FOREACH(ref, &j->artifact_history, entry) {
+ if (strcmp(ref->id, id) != 0)
+ continue;
+ LIST_REMOVE(ref, entry);
+ free(ref);
+ return (0);
+ }
+ return (-1);
+}
+
+static size_t
+job_history_count(const struct job *j)
+{
+ const struct job_artifact_ref *ref;
+ size_t n = 0;
+
+ LIST_FOREACH(ref, &j->artifact_history, entry)
+ n++;
+ return (n);
+}
+
+static int
+job_artifact_id_zero(const char id[ARTIFACT_ID_HEX_LEN + 1])
+{
+ static const char zero[ARTIFACT_ID_HEX_LEN + 1];
+
+ return (memcmp(id, zero, sizeof(zero)) == 0);
+}
+
+static struct job_artifact_tombstone *
+job_tombstone_find(struct joblist *jl, int job_id, const char *id)
+{
+ struct job_artifact_tombstone *tomb;
+
+ LIST_FOREACH(tomb, &jl->tombstones, entry)
+ if (tomb->job_id == job_id && strcmp(tomb->id, id) == 0)
+ return (tomb);
+ return (NULL);
+}
+
+static int
+job_tombstone_add(struct joblist *jl, int job_id, const char *id,
+ uint32_t correlation, int drop_pending)
+{
+ struct job_artifact_tombstone *tomb;
+
+ if ((!artifact_id_wire_valid(id) &&
+ !(drop_pending && job_artifact_id_zero(id))) ||
+ jl->tombstone_count >=
+ jl->tombstone_limit || job_tombstone_find(jl, job_id, id) != NULL ||
+ (drop_pending && correlation == 0))
+ return (-1);
+ tomb = xcalloc(1, sizeof(*tomb));
+ tomb->job_id = job_id;
+ tomb->correlation = correlation;
+ tomb->drop_pending = drop_pending;
+ strlcpy(tomb->id, id, sizeof(tomb->id));
+ LIST_INSERT_HEAD(&jl->tombstones, tomb, entry);
+ jl->tombstone_count++;
+ return (0);
+}
+
+static void
+job_tombstone_remove(struct joblist *jl, struct job_artifact_tombstone *tomb)
+{
+ LIST_REMOVE(tomb, entry);
+ free(tomb);
+ jl->tombstone_count--;
+}
+
+static void
+job_tombstones_remove_job(struct joblist *jl, int job_id)
+{
+ struct job_artifact_tombstone *tomb, *next;
+
+ for (tomb = LIST_FIRST(&jl->tombstones); tomb != NULL; tomb = next) {
+ next = LIST_NEXT(tomb, entry);
+ if (tomb->job_id == job_id)
+ job_tombstone_remove(jl, tomb);
+ }
+}
+
struct joblist *
job_list_new(void)
{
struct joblist *jl;
jl = xcalloc(1, sizeof(*jl));
+ LIST_INIT(&jl->tombstones);
jl->next_id = 1;
jl->cap = DEFAULT_CAP;
+ jl->record_limit = DEFAULT_CAP + JOB_COMPLETED_RECORD_MAX;
+ jl->tombstone_limit = JOB_TOMBSTONE_MAX;
+ jl->ownership_limit = JOB_OWNERSHIP_MAX;
return (jl);
}
job_set_cap(struct joblist *jl, int cap)
{
jl->cap = cap;
+ jl->record_limit = (size_t)cap + JOB_COMPLETED_RECORD_MAX;
}
+void
+job_selftest_set_ownership_limit(struct joblist *jl, size_t limit)
+{
+ if (jl == NULL || limit == 0 || limit > JOB_OWNERSHIP_MAX)
+ abort();
+ jl->ownership_limit = limit;
+}
+
static struct job *
job_find(struct joblist *jl, int id)
{
}
}
+static size_t
+job_record_count(struct joblist *jl)
+{
+ struct job *j;
+ size_t n = 0;
+
+ for (j = jl->head; j != NULL; j = j->next)
+ n++;
+ return (n);
+}
+
+static size_t
+job_ownership_count(struct joblist *jl)
+{
+ struct job *j;
+ size_t n = jl->tombstone_count;
+
+ for (j = jl->head; j != NULL; j = j->next) {
+ n += job_history_count(j);
+ if (j->artifact_managed)
+ n++;
+ }
+ return (n);
+}
+
static void
+job_destroy(struct job *j)
+{
+ job_stop(j);
+ if (j->fd != -1)
+ close(j->fd);
+ if (j->life_fd != -1)
+ close(j->life_fd);
+ if (j->status_fd != -1)
+ close(j->status_fd);
+ job_spool_close(j);
+ artifact_capture_free(&j->capture);
+ buf_free(&j->pending_write);
+ job_history_free(j);
+ free(j);
+}
+
+static int
+job_archive_possible(struct joblist *jl, const struct job *j,
+ int drop_current)
+{
+ const struct job_artifact_ref *ref;
+ size_t needed = job_history_count(j) + (drop_current ? 1 : 0);
+
+ if (jl->tombstone_count > jl->tombstone_limit ||
+ needed > jl->tombstone_limit - jl->tombstone_count)
+ return (0);
+ LIST_FOREACH(ref, &j->artifact_history, entry)
+ if (!artifact_id_wire_valid(ref->id) ||
+ job_tombstone_find(jl, j->id, ref->id) != NULL)
+ return (0);
+ if (drop_current && (!j->artifact_managed || j->artifact_correlation == 0 ||
+ (j->artifact_available ? !artifact_id_wire_valid(j->artifact_id) :
+ !job_artifact_id_zero(j->artifact_id)) ||
+ job_tombstone_find(jl, j->id, j->artifact_id) != NULL))
+ return (0);
+ return (1);
+}
+
+static int
+job_archive(struct joblist *jl, struct job **link, int drop_current)
+{
+ struct job_artifact_ref *ref;
+ struct job *j = *link;
+
+ if (!job_archive_possible(jl, j, drop_current))
+ return (0);
+ LIST_FOREACH(ref, &j->artifact_history, entry)
+ if (job_tombstone_add(jl, j->id, ref->id, 0, 0) == -1)
+ abort();
+ if (drop_current && job_tombstone_add(jl, j->id, j->artifact_id,
+ j->artifact_correlation, 1) == -1)
+ abort();
+ *link = j->next;
+ job_destroy(j);
+ return (1);
+}
+
+static void
+job_prune_reported(struct joblist *jl)
+{
+ struct job **link, *j;
+
+ for (link = &jl->head; (j = *link) != NULL; ) {
+ if (!j->pending && j->reported_terminal &&
+ !j->artifact_available &&
+ job_archive(jl, link, 0))
+ continue;
+ link = &j->next;
+ }
+}
+
+static int
+job_prune_completed_one(struct joblist *jl)
+{
+ struct job **link, **oldest_link = NULL, *j;
+
+ for (link = &jl->head; (j = *link) != NULL; link = &j->next) {
+ int drop_current = j->artifact_managed;
+
+ if (j->pending || j->artifact_settling || !j->exited || j->fd != -1 ||
+ j->artifact_fd != -1 || j->pending_write.len != 0 ||
+ !job_archive_possible(jl, j, drop_current))
+ continue;
+ if (oldest_link == NULL || j->id < (*oldest_link)->id)
+ oldest_link = link;
+ }
+ if (oldest_link == NULL)
+ return (0);
+ return (job_archive(jl, oldest_link,
+ (*oldest_link)->artifact_managed));
+}
+
+int
+job_selftest_archive_completed(struct joblist *jl)
+{
+ struct job *j;
+
+ if (jl == NULL)
+ abort();
+ for (j = jl->head; j != NULL; j = j->next) {
+ if (j->pending || j->artifact_settling || j->fd != -1 ||
+ j->artifact_fd != -1 || j->pending_write.len != 0)
+ continue;
+ job_stop(j);
+ break;
+ }
+ return (job_prune_completed_one(jl));
+}
+
+static int
+job_make_record_room(struct joblist *jl)
+{
+ job_prune_reported(jl);
+ while (job_record_count(jl) >= jl->record_limit)
+ if (!job_prune_completed_one(jl))
+ return (-1);
+ return (0);
+}
+
+static void
job_fail_segment(struct job *j, int storage_error, int unavailable)
{
if (storage_error)
job_spool_close(j);
}
+static void
+job_cap_segment(struct job *j)
+{
+ j->cap_reached = 1;
+ if (!j->exited)
+ (void)killpg(j->pgid, SIGKILL);
+ if (j->fd != -1) {
+ close(j->fd);
+ j->fd = -1;
+ }
+}
+
/* Flush the bounded partial write before reading another producer byte.
* 1 means empty, 0 means pipe backpressure, -1 means storage failure. */
static int
ssize_t n;
while (j->pending_write.len > 0) {
+#ifdef JOB_TESTING
+ if (test_backpressure_at_boundary &&
+ j->capture.retained == ARTIFACT_RAW_MAX) {
+ test_backpressure_at_boundary = 0;
+ errno = EAGAIN;
+ return (0);
+ }
+#endif
n = write(j->artifact_fd, j->pending_write.data,
j->pending_write.len);
if (n > 0) {
want = sizeof(chunk);
if (want > budget - drained)
want = budget - drained;
+ want = artifact_capture_read_limit(&j->capture, want);
+ if (want == 0) {
+ job_cap_segment(j);
+ break;
+ }
n = read(j->fd, chunk, want);
if (n == -1) {
if (errno == EINTR)
job_fail_segment(j, 1, 0);
break;
}
+ if (capture_rc == 1)
+ job_cap_segment(j);
keep = (size_t)j->capture.retained - before;
if (j->artifact_available && keep > 0) {
if (j->pending_write.len > JOB_WRITE_PENDING_MAX - keep) {
break;
}
if (capture_rc == 1) {
- j->cap_reached = 1;
- job_fail_segment(j, 0, 0);
+ if (j->pending_write.len == 0)
+ job_spool_close(j);
break;
}
}
{
struct job *j, *candidate;
size_t total = 0;
+ size_t per_job_limit = JOB_PUMP_PER_JOB;
+ size_t total_limit = JOB_PUMP_TOTAL;
int cursor = jl->pump_cursor, visits = 0, count = 0;
- for (j = jl->head; j != NULL; j = j->next)
+ for (j = jl->head; j != NULL; j = j->next) {
count++;
- while (visits++ < count && total < JOB_PUMP_TOTAL) {
+#ifdef JOB_TESTING
+ j->test_pump_bytes = 0;
+#endif
+ }
+#ifdef JOB_TESTING
+ per_job_limit = test_pump_per_job;
+ total_limit = test_pump_total;
+ jl->test_pump_total = 0;
+#endif
+ while (visits++ < count && total < total_limit) {
+ size_t drained;
+
candidate = NULL;
for (j = jl->head; j != NULL; j = j->next)
if (j->id > cursor &&
}
if (candidate == NULL)
break;
- total += job_drain(candidate,
- JOB_PUMP_TOTAL - total < JOB_PUMP_PER_JOB ?
- JOB_PUMP_TOTAL - total : JOB_PUMP_PER_JOB);
+ drained = job_drain(candidate,
+ total_limit - total < per_job_limit ?
+ total_limit - total : per_job_limit);
+ total += drained;
+#ifdef JOB_TESTING
+ candidate->test_pump_bytes = drained;
+#endif
job_reap(candidate);
cursor = candidate->id;
jl->pump_cursor = cursor;
}
+#ifdef JOB_TESTING
+ jl->test_pump_total = total;
+#endif
}
static int
j->artifact_managed = ctx->artifact_mode;
j->artifact_available = ctx->artifact_mode && ctx->artifact_available;
+ j->artifact_settling = 0;
+ j->artifact_correlation = ctx->artifact_mode ?
+ ctx->artifact_correlation : 0;
j->artifact_fd = -1;
+ if (ctx->artifact_mode && j->artifact_correlation == 0)
+ return (-1);
if (j->artifact_available) {
- if ((flags = fcntl(ctx->artifact_fd, F_GETFL)) == -1 ||
+ if (!artifact_id_wire_valid(ctx->artifact_id) ||
+ job_history_has(j, ctx->artifact_id) ||
+ (flags = fcntl(ctx->artifact_fd, F_GETFL)) == -1 ||
fcntl(ctx->artifact_fd, F_SETFL, flags | O_NONBLOCK) == -1)
return (-1);
j->artifact_fd = ctx->artifact_fd;
ctx->artifact_fd = -1;
strlcpy(j->artifact_id, ctx->artifact_id,
sizeof(j->artifact_id));
- }
+ } else
+ memset(j->artifact_id, 0, sizeof(j->artifact_id));
if (ctx->artifact_mode)
ctx->artifact_spool_adopted = 1;
return (0);
free(cmd);
return;
}
+ if (job_make_record_room(jl) == -1) {
+ buf_addstr(out, "shell_bg: background-job settlement backlog; "
+ "poll an existing completed job first");
+ *is_error = 1;
+ free(cmd);
+ return;
+ }
+ if (ctx->artifact_mode &&
+ job_ownership_count(jl) >= jl->ownership_limit) {
+ buf_addstr(out, "shell_bg: Artifact settlement backlog; "
+ "settle an earlier background spool first");
+ *is_error = 1;
+ free(cmd);
+ return;
+ }
if (pipe(pipefd) == -1) {
buf_addf(out, "shell_bg: pipe: %s", strerror(errno));
*is_error = 1;
free(cmd);
j = xcalloc(1, sizeof(*j));
+ LIST_INIT(&j->artifact_history);
j->id = (int)jl->next_id++;
j->pid = sp.pid;
j->pgid = sp.pgid;
}
j->next = jl->head;
jl->head = j;
+ ctx->artifact_job_id = j->id;
buf_addf(out, "started background job %d", j->id);
}
}
static int
-job_flush_blocking(struct job *j)
+job_flush_settle(struct job *j)
{
- int flags;
+ struct pollfd pfd;
+ int rc;
if (!j->artifact_available || j->artifact_fd == -1)
return (0);
- if ((flags = fcntl(j->artifact_fd, F_GETFL)) == -1 ||
- fcntl(j->artifact_fd, F_SETFL, flags & ~O_NONBLOCK) == -1) {
- job_fail_segment(j, 1, 0);
- return (-1);
- }
while (j->pending_write.len > 0) {
- ssize_t n = write(j->artifact_fd, j->pending_write.data,
- j->pending_write.len);
-
- if (n > 0) {
- buf_drain(&j->pending_write, (size_t)n);
- continue;
+ if (tool_cancel_requested())
+ return (1);
+ rc = job_flush(j);
+ if (rc == 1)
+ return (0);
+ if (rc == -1)
+ return (-1);
+ pfd.fd = j->artifact_fd;
+ pfd.events = POLLOUT;
+ pfd.revents = 0;
+ rc = poll(&pfd, 1, 100);
+ if (rc == -1) {
+ if (errno == EINTR)
+ continue;
+ goto fail;
}
- if (n == -1 && errno == EINTR)
- continue;
+ if (pfd.revents & (POLLERR | POLLHUP | POLLNVAL))
+ goto fail;
+ }
+ return (0);
+fail:
+ job_fail_segment(j, 1, 0);
+ return (-1);
+}
+
+/* Settle the old spool while consuming no more than the single entry
+ * snapshot. Cancellation leaves both producer and spool ownership intact. */
+static int
+job_freeze_snapshot(struct job *j, size_t snapshot)
+{
+ size_t drained = 0, n;
+ int rc;
+
+ for (;;) {
+ rc = job_flush_settle(j);
+ if (rc != 0)
+ return (rc);
+ if (drained == snapshot || j->fd == -1 || j->cap_reached ||
+ j->storage_error || j->storage_unavailable)
+ return (0);
+ n = job_drain(j, snapshot - drained);
+ drained += n;
+ if (n == 0 && j->pending_write.len == 0)
+ return (0);
+ }
+}
+
+/* Freeze only bytes already queued by the producer when shell_output entered.
+ * The per-job pump budget is also the synchronous poll budget; bytes queued
+ * later remain in the producer pipe for the replacement segment. */
+static int
+job_poll_snapshot(struct job *j, size_t *snapshot)
+{
+ int bytes = 0;
+
+ *snapshot = 0;
+ if (j->fd == -1)
+ return (0);
+ if (ioctl(j->fd, FIONREAD, &bytes) == -1 || bytes < 0) {
job_fail_segment(j, 1, 0);
return (-1);
}
+ *snapshot = (size_t)bytes;
+ if (*snapshot > JOB_PUMP_PER_JOB)
+ *snapshot = JOB_PUMP_PER_JOB;
+ *snapshot = artifact_capture_read_limit(&j->capture, *snapshot);
return (0);
}
+/* Reaping fixes the producer set: the supervisor has killed the process group
+ * and closed every writer. FIONREAD==0 plus POLLHUP can therefore close EOF
+ * without consuming a byte that belongs to a later poll segment. */
static void
+job_close_reaped_eof(struct job *j)
+{
+ struct pollfd pfd;
+ int bytes = 0, rc;
+
+ if (!j->exited || j->fd == -1)
+ return;
+ if (ioctl(j->fd, FIONREAD, &bytes) == -1 || bytes < 0) {
+ job_fail_segment(j, 1, 0);
+ return;
+ }
+ if (bytes != 0)
+ return;
+ pfd.fd = j->fd;
+ pfd.events = POLLIN;
+ pfd.revents = 0;
+ rc = poll(&pfd, 1, 0);
+ if (rc == -1) {
+ if (errno != EINTR)
+ job_fail_segment(j, 1, 0);
+ return;
+ }
+ if (rc == 1 && (pfd.revents & POLLHUP)) {
+ close(j->fd);
+ j->fd = -1;
+ if (j->pending_write.len == 0)
+ job_spool_close(j);
+ }
+}
+
+static void
job_result_terminal(struct job *j, struct tool_ctx *ctx, int *is_error)
{
ctx->artifact_exit_status = 0;
int obj, struct buf *out, int *is_error)
{
struct job *j;
- int id;
+ size_t snapshot = 0;
+ int id, terminal = 0;
if (job_id_arg(js, obj, "shell_output", out, is_error, &id) == -1)
return;
*is_error = 1;
return;
}
- (void)job_drain(j, JOB_PUMP_PER_JOB);
+ if (j->artifact_settling) {
+ buf_addf(out, "shell_output: job %d is awaiting Artifact "
+ "settlement", id);
+ *is_error = 1;
+ return;
+ }
+ ctx->artifact_job_id = j->id;
+ (void)job_poll_snapshot(j, &snapshot);
+ if (job_freeze_snapshot(j, snapshot) == 1)
+ return;
job_reap(j);
- if (j->exited)
- (void)job_drain(j, JOB_PUMP_PER_JOB);
+ job_close_reaped_eof(j);
if (ctx->artifact_mode) {
- int adopt;
+ int adopt, ownership_backlog = 0;
- (void)job_flush_blocking(j);
+ adopt = !j->cap_reached && !j->storage_error &&
+ !j->storage_unavailable && (!j->exited || j->fd != -1);
+ if (adopt &&
+ job_ownership_count(jl) >= jl->ownership_limit) {
+ /* Moving current to history is ownership-neutral, but adopting
+ * this replacement would add another unresolved exact key. */
+ ownership_backlog = 1;
+ adopt = 0;
+ job_stop(j);
+ if (j->fd != -1) {
+ close(j->fd);
+ j->fd = -1;
+ }
+ }
+ if (j->artifact_available && !ownership_backlog &&
+ job_history_add(j, j->artifact_id) == -1)
+ abort();
job_spool_close(j);
ctx->artifact_captured = 1;
ctx->artifact_observed = j->capture.observed;
strlcpy(ctx->artifact_result_id, j->artifact_id,
sizeof(ctx->artifact_result_id));
job_result_terminal(j, ctx, is_error);
+ if (ownership_backlog) {
+ ctx->artifact_termination = ARTIFACT_TERM_CANCELLED;
+ *is_error = 1;
+ }
artifact_capture_free(&ctx->artifact_capture);
ctx->artifact_capture = j->capture;
artifact_capture_init(&j->capture);
- adopt = !j->cap_reached && !j->storage_error &&
- !j->storage_unavailable && (!j->exited || j->fd != -1);
+ terminal = !adopt;
j->cap_reached = 0;
j->storage_error = 0;
j->storage_unavailable = 0;
- j->artifact_available = 0;
- j->artifact_managed = 0;
- memset(j->artifact_id, 0, sizeof(j->artifact_id));
+ if (ownership_backlog && j->artifact_available) {
+ /* The already-published current key is the bounded settlement
+ * slot. RELEASE/CANCEL retires it; no replacement is adopted. */
+ j->artifact_settling = 1;
+ } else {
+ j->artifact_available = 0;
+ j->artifact_settling = 0;
+ j->artifact_managed = 0;
+ j->artifact_correlation = 0;
+ memset(j->artifact_id, 0, sizeof(j->artifact_id));
+ }
if (adopt) {
ctx->artifact_spool_detach = !j->pending;
if (job_adopt_stream(j, ctx) == -1) {
job_stop(j);
+ if (j->fd != -1) {
+ close(j->fd);
+ j->fd = -1;
+ }
+ j->artifact_available = 0;
+ j->artifact_settling = 0;
+ j->artifact_managed = 0;
+ j->artifact_correlation = 0;
ctx->artifact_termination =
ARTIFACT_TERM_STORAGE_ERROR;
*is_error = 1;
+ terminal = 1;
}
}
} else {
"some was dropped]\n");
artifact_capture_reset(&j->capture);
j->cap_reached = 0;
+ terminal = j->exited && j->fd == -1;
}
-
- if (ctx->artifact_mode)
+ j->reported_terminal = terminal;
+ if (ctx->artifact_mode) {
+ job_prune_reported(jl);
return;
+ }
if (j->exited) {
if (WIFEXITED(j->status))
buf_addf(out, "\n[fugu: job %d exited, status %d]\n",
id, WTERMSIG(j->status));
} else
buf_addf(out, "\n[fugu: job %d running]\n", id);
+ job_prune_reported(jl);
}
void
}
}
-void
-job_artifact_cancel(struct joblist *jl, const char *id)
+int
+job_artifact_cancel(struct joblist *jl, int job_id, const char *id)
{
+ struct job_artifact_tombstone *tomb;
struct job *j;
+ int live_match;
if (!artifact_id_wire_valid(id))
- return;
- for (j = jl->head; j != NULL; j = j->next) {
- if (!j->artifact_available || strcmp(j->artifact_id, id) != 0)
- continue;
+ return (-1);
+ j = job_find(jl, job_id);
+ live_match = j != NULL && ((j->artifact_available &&
+ strcmp(j->artifact_id, id) == 0) || job_history_has(j, id));
+ tomb = job_tombstone_find(jl, job_id, id);
+ if (!live_match && (tomb == NULL ||
+ (tomb->drop_pending && job_artifact_id_zero(tomb->id))))
+ return (-1);
+ if (live_match) {
job_fail_segment(j, 1, 0);
- return;
+ j->artifact_available = 0;
+ j->artifact_settling = 0;
+ j->artifact_managed = 0;
+ j->artifact_correlation = 0;
+ memset(j->artifact_id, 0, sizeof(j->artifact_id));
+ job_history_free(j);
+ job_tombstones_remove_job(jl, job_id);
+ job_prune_reported(jl);
+ return (0);
}
+ job_tombstones_remove_job(jl, job_id);
+ return (0);
}
int
+job_artifact_release(struct joblist *jl, int job_id, const char *id)
+{
+ struct job_artifact_tombstone *tomb;
+ struct job *j;
+ int rc;
+
+ if (!artifact_id_wire_valid(id))
+ return (-1);
+ j = job_find(jl, job_id);
+ if (j != NULL && j->artifact_settling && j->artifact_available &&
+ strcmp(j->artifact_id, id) == 0) {
+ j->artifact_available = 0;
+ j->artifact_settling = 0;
+ j->artifact_managed = 0;
+ j->artifact_correlation = 0;
+ memset(j->artifact_id, 0, sizeof(j->artifact_id));
+ job_prune_reported(jl);
+ return (0);
+ }
+ if (j != NULL && (rc = job_history_remove(j, id)) == 0) {
+ job_prune_reported(jl);
+ return (0);
+ }
+ tomb = job_tombstone_find(jl, job_id, id);
+ if (tomb == NULL || tomb->drop_pending)
+ return (-1);
+ job_tombstone_remove(jl, tomb);
+ return (0);
+}
+
+int
+job_artifact_drop_next(struct joblist *jl, uint32_t *correlation,
+ struct artifact_spool_drop *drop)
+{
+ struct job_artifact_tombstone *tomb;
+
+ if (correlation == NULL || drop == NULL)
+ return (-1);
+ LIST_FOREACH(tomb, &jl->tombstones, entry) {
+ if (!tomb->drop_pending || tomb->drop_sent)
+ continue;
+ if (tomb->correlation == 0 || tomb->job_id <= 0)
+ return (-1);
+ memset(drop, 0, sizeof(*drop));
+ drop->job_id = (uint32_t)tomb->job_id;
+ strlcpy(drop->artifact_id, tomb->id,
+ sizeof(drop->artifact_id));
+ *correlation = tomb->correlation;
+ tomb->drop_sent = 1;
+ return (1);
+ }
+ return (0);
+}
+
+int
+job_artifact_drop_ack(struct joblist *jl, uint32_t correlation,
+ const struct artifact_spool_drop *drop)
+{
+ struct job_artifact_tombstone *tomb;
+
+ if (correlation == 0 || drop == NULL || drop->job_id == 0 ||
+ drop->job_id > INT_MAX || (!artifact_id_wire_valid(drop->artifact_id) &&
+ !job_artifact_id_zero(drop->artifact_id)))
+ return (-1);
+ tomb = job_tombstone_find(jl, (int)drop->job_id, drop->artifact_id);
+ if (tomb == NULL)
+ return (-1);
+ if (!tomb->drop_pending || !tomb->drop_sent ||
+ tomb->correlation != correlation)
+ return (-1);
+ job_tombstone_remove(jl, tomb);
+ return (0);
+}
+
+int
job_turn_begin(struct joblist *jl)
{
if (jl->turn_open)
for (j = jl->head; j != NULL; j = j->next)
j->pending = 0;
jl->turn_open = 0;
+ job_prune_reported(jl);
return (0);
}
job_spool_close(j);
artifact_capture_free(&j->capture);
buf_free(&j->pending_write);
+ job_history_free(j);
*link = j->next;
free(j);
}
job_list_free(struct joblist *jl)
{
struct job *j, *n;
+ struct job_artifact_tombstone *tomb;
if (jl == NULL)
return;
job_spool_close(j);
artifact_capture_free(&j->capture);
buf_free(&j->pending_write);
+ job_history_free(j);
free(j);
}
+ while ((tomb = LIST_FIRST(&jl->tombstones)) != NULL) {
+ LIST_REMOVE(tomb, entry);
+ free(tomb);
+ }
free(jl);
}
blob - 725cc0a2c5451c676c51809104829f76d81ed03f
blob + 8e3b9e77a89f6c8b6d7171161b8dbd1721937fad
--- src/fugu-tool/job.h
+++ src/fugu-tool/job.h
#ifndef JOB_H
#define JOB_H
+#include <stdint.h>
+
#include "buf.h"
+#include "proto.h"
/*
* Background shell jobs (behavior.md section 5): shell_bg starts a
/* the max_bg_jobs gate; 0 (the default here) means unconfigured */
void job_set_cap(struct joblist *, int cap);
+/* Regression-only bound selected by fugu-tool under guarded selftest mode. */
+void job_selftest_set_ownership_limit(struct joblist *, size_t);
+int job_selftest_archive_completed(struct joblist *);
void job_start(struct joblist *, struct tool_ctx *, const struct json *,
int obj, struct buf *out, int *is_error);
/* drain every running job's pipe into its ring (called opportunistically) */
void job_pump(struct joblist *);
-void job_artifact_cancel(struct joblist *, const char *);
+int job_artifact_cancel(struct joblist *, int, const char *);
+int job_artifact_release(struct joblist *, int, const char *);
+int job_artifact_drop_next(struct joblist *, uint32_t *,
+ struct artifact_spool_drop *);
+int job_artifact_drop_ack(struct joblist *, uint32_t,
+ const struct artifact_spool_drop *);
/* kill and reap every job (turn abandon, session end) */
void job_kill_all(struct joblist *);
int job_turn_commit(struct joblist *);
int job_turn_abort(struct joblist *);
+#ifdef JOB_TESTING
+struct job_test_state {
+ uint64_t observed;
+ uint64_t retained;
+ size_t pending_bytes;
+ int cap_reached;
+ int exited;
+ int producer_open;
+ int artifact_open;
+ int storage_error;
+ int storage_unavailable;
+};
+
+void job_test_backpressure_at_boundary(void);
+int job_test_boundary_pending(struct joblist *, int);
+int job_test_storage_unavailable(struct joblist *, int);
+size_t job_test_pending_bytes(struct joblist *, int);
+size_t job_test_output_bytes(struct joblist *, int);
+void job_test_set_pump_limits(size_t, size_t);
+void job_test_reset_pump_limits(void);
+size_t job_test_pump_per_job_limit(void);
+size_t job_test_pump_total_limit(void);
+size_t job_test_last_pump_total(struct joblist *);
+size_t job_test_last_pump_bytes(struct joblist *, int);
+int job_test_state(struct joblist *, int, struct job_test_state *);
+void job_test_set_record_limit(struct joblist *, size_t);
+void job_test_set_tombstone_limit(struct joblist *, size_t);
+size_t job_test_record_count(struct joblist *);
+size_t job_test_tombstone_count(struct joblist *);
+void job_test_set_ownership_limit(struct joblist *, size_t);
+size_t job_test_ownership_count(struct joblist *);
+#endif
+
#endif /* JOB_H */
blob - 7836e330da2f4510de02c7738bde3617d62e0c1d
blob + 20e8d5ffe78a9765287b58c24e371c50dba899ef
--- src/fugu-tool/main.c
+++ src/fugu-tool/main.c
#include <stdlib.h>
#include <string.h>
#include <syslog.h>
+#include <time.h>
#include <unistd.h>
#include "log.h"
#include "tools.h"
#include "agent.h"
#include "artifact_result.h"
+#include "artifact_pipe.h"
+#include "artifact_scope.h"
static int readonly; /* -r: the subagent role, always read-only */
+static long long selftest_rotation_delay_ms;
+static long long selftest_drop_delay_ms;
+static int selftest_archive_pending;
+static int selftest_rotation_trace;
+static int selftest_drop_trace;
struct tool_state {
+ struct worker *worker;
struct tool_ctx *ctx;
char name[FUGU_TOOL_NAME_MAX];
int have_name;
* a clean shutdown that would otherwise orphan them into init. */
static struct joblist *jobs_atexit;
+static long long
+selftest_delay(const char *name)
+{
+ const char *value, *errstr;
+ long long ms;
+
+ if (issetugid() || getenv("FUGU_SELFTEST") == NULL ||
+ (value = getenv(name)) == NULL)
+ return (0);
+ ms = strtonum(value, 1, 5000, &errstr);
+ if (errstr != NULL)
+ fatalx("fugu-tool: invalid %s", name);
+ return (ms);
+}
+
+static int
+selftest_flag(const char *name)
+{
+ const char *value;
+
+ if (issetugid() || getenv("FUGU_SELFTEST") == NULL ||
+ (value = getenv(name)) == NULL)
+ return (0);
+ if (strcmp(value, "1") != 0)
+ fatalx("fugu-tool: invalid %s", name);
+ return (1);
+}
+
static void
+selftest_pause(long long *milliseconds, const char *marker)
+{
+ struct timespec delay;
+ long long ms;
+
+ if ((ms = *milliseconds) == 0)
+ return;
+ *milliseconds = 0;
+ log_warnx("%s", marker);
+ delay.tv_sec = (time_t)(ms / 1000);
+ delay.tv_nsec = (long)(ms % 1000) * 1000000L;
+ (void)nanosleep(&delay, NULL);
+}
+
+static int
+spool_drop_id_valid(const char id[FUGU_ARTIFACT_ID_BUFSZ])
+{
+ static const char zero[FUGU_ARTIFACT_ID_BUFSZ];
+
+ return (artifact_id_wire_valid(id) ||
+ memcmp(id, zero, sizeof(zero)) == 0);
+}
+
+static void
+send_spool_drops(struct worker *w, struct tool_state *ts)
+{
+ struct artifact_spool_drop drop;
+ uint32_t correlation;
+ int rc;
+
+ for (;;) {
+ correlation = 0;
+ memset(&drop, 0, sizeof(drop));
+ rc = job_artifact_drop_next(ts->ctx->jobs, &correlation, &drop);
+ if (rc == 0)
+ return;
+ if (rc == -1 || correlation == 0 || drop.job_id == 0 ||
+ !spool_drop_id_valid(drop.artifact_id))
+ fatalx("fugu-tool: invalid Artifact spool drop");
+ imsgev_send(&w->iev, FUGU_IMSG_TOOL_ARTIFACT_DROP, correlation,
+ &drop, sizeof(drop));
+ selftest_pause(&selftest_drop_delay_ms,
+ "fugu-tool: selftest Artifact DROP queued");
+ }
+}
+
+static void
terminate_signal(int sig)
{
(void)sig;
(void)fd;
(void)ev;
- if (ts->ctx != NULL)
+ if (ts->ctx != NULL) {
job_pump(ts->ctx->jobs);
+ if (selftest_archive_pending &&
+ job_selftest_archive_completed(ts->ctx->jobs) == 1)
+ selftest_archive_pending = 0;
+ send_spool_drops(ts->worker, ts);
+ }
}
/*
}
ts->ctx = tool_ctx_new(write_ok, net_ok);
+ ts->worker = w;
job_set_cap(ts->ctx->jobs, (int)ts->cap);
+ if (selftest_rotation_delay_ms > 0)
+ job_selftest_set_ownership_limit(ts->ctx->jobs, 1);
/* kill our background jobs when we exit, and keep their output
* draining between polls (event_init ran before this dispatch) */
sizeof(*result));
}
+/* Publish every completed shell_output freeze independently of the ordinary
+ * result. replacement_adopted distinguishes a swap from a terminal/error
+ * freeze; no frame means the Tool call did not mutate spool ownership. The
+ * Coordinator may abandon the ordinary result after SIGUSR1, but this frame
+ * still proves exactly which old ownership was frozen. */
+static void
+send_spool_rotation(struct worker *w, const struct tool_state *ts, uint32_t id)
+{
+ struct artifact_spool_rotate rotate;
+
+ if (!ts->ctx->artifact_captured || ts->ctx->artifact_job_id <= 0)
+ return;
+ if (id == 0)
+ fatalx("fugu-tool: invalid Artifact spool rotation");
+ if (!ts->ctx->artifact_spool_adopted && ts->ctx->artifact_spool_detach)
+ fatalx("fugu-tool: detached unadopted Artifact spool");
+ memset(&rotate, 0, sizeof(rotate));
+ rotate.job_id = (uint32_t)ts->ctx->artifact_job_id;
+ rotate.replacement_adopted = ts->ctx->artifact_spool_adopted;
+ rotate.detach = ts->ctx->artifact_spool_detach;
+ if (ts->ctx->artifact_result_available)
+ strlcpy(rotate.frozen_id, ts->ctx->artifact_result_id,
+ sizeof(rotate.frozen_id));
+ if (ts->ctx->artifact_spool_adopted && ts->ctx->artifact_available)
+ strlcpy(rotate.replacement_id, ts->artifact.id,
+ sizeof(rotate.replacement_id));
+ if (selftest_rotation_delay_ms > 0) {
+ imsgev_send_blocking(&w->iev, FUGU_IMSG_TOOL_ARTIFACT_ROTATE, id,
+ &rotate, sizeof(rotate));
+ selftest_pause(&selftest_rotation_delay_ms,
+ "fugu-tool: selftest frame0 Artifact rotation published");
+ return;
+ }
+ imsgev_send(&w->iev, FUGU_IMSG_TOOL_ARTIFACT_ROTATE, id, &rotate,
+ sizeof(rotate));
+}
+
/* Replace raw in-scope bytes with the single generated provider document.
* artifact_result emits id first exactly when the staging object is needed. */
static void
result->is_error = *is_error;
result->spool_adopted = 1;
result->spool_detach = ts->ctx->artifact_spool_detach;
+ result->background_job_id = ts->ctx->artifact_job_id;
if (ts->ctx->artifact_available)
strlcpy(result->artifact_id, ts->artifact.id,
sizeof(result->artifact_id));
memset(&in, 0, sizeof(in));
in.raw = NULL;
in.raw_len = ts->ctx->artifact_raw_len;
- in.id = ts->ctx->artifact_captured ?
+ /* A foreground capture writes the stream brokered for this call. A
+ * shell_output capture instead describes the frozen prior segment and
+ * therefore uses only that job's recorded result id. */
+ in.id = ts->ctx->artifact_captured && ts->ctx->artifact_job_id > 0 ?
(ts->ctx->artifact_result_available ?
ts->ctx->artifact_result_id : NULL) :
(ts->ctx->artifact_available ? ts->artifact.id : NULL);
result->artifact = 1;
result->keep_artifact = keep;
result->artifact_cap_reached = ts->ctx->artifact_cap_reached;
+ result->spool_frozen = ts->ctx->artifact_captured &&
+ ts->ctx->artifact_job_id > 0;
result->spool_adopted = ts->ctx->artifact_spool_adopted;
result->spool_detach = ts->ctx->artifact_spool_detach;
+ result->background_job_id = ts->ctx->artifact_job_id;
result->termination = ts->ctx->artifact_termination;
result->observed_bytes = ts->ctx->artifact_observed;
result->retained_bytes = ts->ctx->artifact_raw_len;
switch (imsg_get_type(imsg)) {
case FUGU_IMSG_TOOL_ARTIFACT_STREAM: {
struct artifact_stream stream;
- struct stat st;
- int fd = -1, flags, fdflags;
+ int fd = -1;
- if (readonly || ts->agent || ts->have_artifact ||
+ if (readonly || ts->agent || ts->have_artifact || ts->have_name ||
+ ts->arg.len != 0 ||
imsg_get_id(imsg) == 0)
fatalx("fugu-tool: unexpected Artifact stream");
if (imsgev_get_fd_payload(imsg, &fd, &payload) == -1) {
memcpy(&stream, p, sizeof(stream));
if ((stream.status == ARTIFACT_STREAM_AVAILABLE &&
(fd == -1 || !artifact_id_wire_valid(stream.id) ||
- fstat(fd, &st) == -1 || !S_ISFIFO(st.st_mode) ||
- (flags = fcntl(fd, F_GETFL)) == -1 ||
- flags != O_WRONLY ||
- (fdflags = fcntl(fd, F_GETFD)) == -1 ||
- (fdflags & FD_CLOEXEC) == 0)) ||
+ !artifact_producer_endpoint_valid(fd))) ||
(stream.status == ARTIFACT_STREAM_UNAVAILABLE &&
(fd != -1 || memcmp(stream.id,
(char[FUGU_ARTIFACT_ID_BUFSZ]){0}, sizeof(stream.id)) != 0)) ||
sizeof(ack));
return (0);
}
- case FUGU_IMSG_TOOL_ARTIFACT_CANCEL: {
+ case FUGU_IMSG_TOOL_ARTIFACT_DROP_ACK: {
+ struct artifact_spool_drop drop;
+ uint32_t correlation = imsg_get_id(imsg);
+
+ if (readonly || ts->agent || correlation == 0 ||
+ imsgev_get_payload(imsg, &payload) == -1 ||
+ payload.data == NULL || payload.len != sizeof(drop))
+ fatalx("fugu-tool: bad Artifact drop acknowledgement");
+ memcpy(&drop, payload.data, sizeof(drop));
+ if (job_artifact_drop_ack(ts->ctx->jobs, correlation, &drop) == -1)
+ fatalx("fugu-tool: stale Artifact drop acknowledgement");
+ return (0);
+ }
+ case FUGU_IMSG_TOOL_ARTIFACT_CANCEL:
+ case FUGU_IMSG_TOOL_ARTIFACT_RELEASE: {
+ struct artifact_job_cancel_ack ack;
struct artifact_record record;
+ uint32_t type = imsg_get_type(imsg), job_id = imsg_get_id(imsg);
+ int rc;
- if (readonly || ts->agent || imsg_get_id(imsg) != 0 ||
+ if (readonly || ts->agent || job_id == 0 || job_id > INT_MAX ||
imsgev_get_payload(imsg, &payload) == -1 ||
payload.data == NULL || payload.len != sizeof(record))
fatalx("fugu-tool: bad Artifact cancellation");
memcpy(&record, payload.data, sizeof(record));
if (!artifact_id_wire_valid(record.id))
fatalx("fugu-tool: malformed Artifact cancellation id");
- job_artifact_cancel(ts->ctx->jobs, record.id);
+ rc = type == FUGU_IMSG_TOOL_ARTIFACT_CANCEL ?
+ job_artifact_cancel(ts->ctx->jobs, (int)job_id, record.id) :
+ job_artifact_release(ts->ctx->jobs, (int)job_id, record.id);
+ if (rc == -1)
+ fatalx("fugu-tool: stale Artifact job ownership");
+ if (type == FUGU_IMSG_TOOL_ARTIFACT_CANCEL) {
+ memset(&ack, 0, sizeof(ack));
+ ack.job_id = job_id;
+ strlcpy(ack.artifact_id, record.id,
+ sizeof(ack.artifact_id));
+ imsgev_send(&w->iev, FUGU_IMSG_TOOL_ARTIFACT_CANCEL_ACK,
+ job_id, &ack, sizeof(ack));
+ if (selftest_drop_trace)
+ log_warnx("fugu-tool: selftest Artifact CANCEL_ACK "
+ "job=%u", job_id);
+ } else if (selftest_rotation_trace) {
+ log_warnx("fugu-tool: selftest Artifact RELEASE job=%u",
+ job_id);
+ }
return (0);
}
case FUGU_IMSG_TOOL_REQ: {
struct tool_req req;
+ int scoped;
if (imsgev_get_payload(imsg, &payload) == -1)
fatal("get TOOL_REQ payload");
p = payload.data;
len = payload.len;
- if (p == NULL || len != sizeof(req) ||
- (ts->have_artifact && imsg_get_id(imsg) != ts->artifact_id) ||
- (!ts->have_artifact && imsg_get_id(imsg) != 0))
+ if (p == NULL || len != sizeof(req) || ts->have_name ||
+ ts->arg.len != 0)
fatalx("fugu-tool: short TOOL_REQ");
memcpy(&req, p, sizeof(req));
req.name[sizeof(req.name) - 1] = '\0';
+ scoped = artifact_capture_tool_name(req.name);
+ if ((!readonly && !ts->agent && !!ts->have_artifact != !!scoped) ||
+ (ts->have_artifact && imsg_get_id(imsg) != ts->artifact_id) ||
+ (!ts->have_artifact && imsg_get_id(imsg) != 0))
+ fatalx("fugu-tool: Artifact scope mismatch");
strlcpy(ts->name, req.name, sizeof(ts->name));
ts->have_name = 1;
buf_reset(&ts->arg);
ts->ctx->artifact_available = ts->have_artifact &&
ts->artifact.status == ARTIFACT_STREAM_AVAILABLE;
ts->ctx->artifact_id = ts->have_artifact ? ts->artifact.id : NULL;
+ ts->ctx->artifact_correlation = ts->have_artifact ?
+ ts->artifact_id : 0;
ts->ctx->artifact_observed = 0;
ts->ctx->artifact_raw_len = 0;
ts->ctx->artifact_termination = ARTIFACT_TERM_EXIT;
ts->ctx->artifact_captured = 0;
tool_exec(ts->ctx, ts->name, ts->arg.data, ts->arg.len,
&out, &is_error);
- if (ts->have_artifact)
+ send_spool_drops(w, ts);
+ if (ts->have_artifact) {
+ send_spool_rotation(w, ts, ts->artifact_id);
serialize_artifact_result(ts, &out, &is_error, &result);
- else
+ } else
result.is_error = is_error;
ts->artifact_fd = ts->ctx->artifact_fd;
if (ts->artifact_fd != -1) {
ts->ctx->artifact_mode = 0;
ts->ctx->artifact_available = 0;
ts->ctx->artifact_id = NULL;
+ ts->ctx->artifact_correlation = 0;
return (0);
}
default:
sigemptyset(&sa.sa_mask);
if (sigaction(SIGTERM, &sa, NULL) == -1)
fatal("sigaction SIGTERM");
+ if (sigaction(SIGUSR1, &sa, NULL) == -1)
+ fatal("sigaction SIGUSR1");
buf_init(&ts.arg);
ts.artifact_fd = -1;
agent_boot_init(&ts.boot);
+ selftest_rotation_delay_ms = selftest_delay(
+ "FUGU_ARTIFACT_FRAME0_DELAY_MS");
+ selftest_drop_delay_ms = selftest_delay("FUGU_ARTIFACT_DROP_DELAY_MS");
+ selftest_archive_pending = selftest_flag(
+ "FUGU_ARTIFACT_ARCHIVE_COMPLETED");
+ selftest_rotation_trace = selftest_rotation_delay_ms > 0;
+ selftest_drop_trace = selftest_drop_delay_ms > 0;
/*
* Transitional pledge. It is a superset of the steady promises;
blob - b1cfd11bd33072fd502be44807cdb2d017a74aaf
blob + 8091e3a43e76e43139bbe49a9f1cfb6292913d93
--- src/fugu-tool/tools.c
+++ src/fugu-tool/tools.c
#define CAP_ARTIFACT_CAP (-5)
static volatile sig_atomic_t active_pgid;
+static volatile sig_atomic_t active_cancelled;
-static int
-artifact_write_all(int fd, const void *data, size_t len)
+enum artifact_write_result {
+ ARTIFACT_WRITE_OK = 0,
+ ARTIFACT_WRITE_ERROR,
+ ARTIFACT_WRITE_TIMEOUT,
+ ARTIFACT_WRITE_CANCELLED
+};
+
+static enum artifact_write_result
+artifact_write_all(int fd, const void *data, size_t len,
+ const struct timespec *deadline)
{
const u_char *p = data;
+ struct pollfd pfd;
ssize_t n;
while (len > 0) {
- do {
- n = write(fd, p, len);
- } while (n == -1 && errno == EINTR);
- if (n <= 0)
- return (-1);
- p += n;
- len -= (size_t)n;
+ if (active_cancelled)
+ return (ARTIFACT_WRITE_CANCELLED);
+ n = write(fd, p, len);
+ if (n > 0) {
+ p += n;
+ len -= (size_t)n;
+ continue;
+ }
+ if (n == -1 && errno == EINTR)
+ continue;
+ if (n == -1 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
+ int left = deadline_ms(deadline);
+
+ if (left == 0)
+ return (ARTIFACT_WRITE_TIMEOUT);
+ pfd.fd = fd;
+ pfd.events = POLLOUT;
+ if (left > 100)
+ left = 100;
+ if (poll(&pfd, 1, left) == -1 && errno != EINTR)
+ return (ARTIFACT_WRITE_ERROR);
+ continue;
+ }
+ return (ARTIFACT_WRITE_ERROR);
}
- return (0);
+ return (ARTIFACT_WRITE_OK);
}
void
{
pid_t pid = (pid_t)active_pgid;
+ active_cancelled = 1;
if (pid > 0)
(void)kill(-pid, SIGKILL);
}
+int
+tool_cancel_requested(void)
+{
+ return (active_cancelled != 0);
+}
+
/* Reap and clear the published process group with SIGTERM blocked, so a
* teardown signal can never target a pid after its group leader was reaped
* and made available for reuse. */
sigemptyset(&block);
sigaddset(&block, SIGTERM);
+ sigaddset(&block, SIGUSR1);
if (sigprocmask(SIG_BLOCK, &block, &oldmask) == -1)
return (-1);
r = waitpid(sp->pid, &supervisor_status, 0);
sigemptyset(&block);
sigaddset(&block, SIGTERM);
+ sigaddset(&block, SIGUSR1);
if (sigprocmask(SIG_BLOCK, &block, &oldmask) == -1)
return (-1);
memset(&si, 0, sizeof(si));
pid_t pid, pgid;
int pipefd[2], status = 0, reaped = 0;
int timed_out = 0, truncated = 0, storage_error = 0;
+ int flags;
size_t capture_cap = ctx->artifact_mode ? ARTIFACT_RAW_MAX :
CAPTURE_CAP;
int tmo = ctx->shell_timeout_ms > 0 ?
ctx->shell_timeout_ms : SHELL_TIMEOUT;
*sig = 0;
+ active_cancelled = 0;
if (pipe(pipefd) == -1) {
result_err(out, is_error, "pipe: %s", strerror(errno));
return (CAP_SPAWNERR);
close(pipefd[1]);
return (CAP_SPAWNERR);
}
+ if (ctx->artifact_mode && ctx->artifact_available &&
+ ((flags = fcntl(ctx->artifact_fd, F_GETFL)) == -1 ||
+ fcntl(ctx->artifact_fd, F_SETFL, flags | O_NONBLOCK) == -1)) {
+ result_err(out, is_error, "artifact fcntl: %s", strerror(errno));
+ close(pipefd[0]);
+ close(pipefd[1]);
+ return (CAP_STORAGE);
+ }
sigemptyset(&block);
sigaddset(&block, SIGTERM);
+ sigaddset(&block, SIGUSR1);
if (sigprocmask(SIG_BLOCK, &block, &oldmask) == -1) {
result_err(out, is_error, "sigprocmask: %s", strerror(errno));
close(pipefd[0]);
for (;;) {
u_char chunk[16384];
ssize_t n;
+ size_t want = sizeof(chunk);
int left = deadline_ms(&deadline), w;
w = 0;
goto capture_done;
continue;
}
- n = read(pipefd[0], chunk, sizeof(chunk));
+ if (ctx->artifact_mode) {
+ want = artifact_capture_read_limit(&ctx->artifact_capture,
+ want);
+ if (want == 0) {
+ truncated = 1;
+ break;
+ }
+ }
+ n = read(pipefd[0], chunk, want);
if (n == -1) {
if (errno == EAGAIN || errno == EINTR)
continue;
(size_t)ctx->artifact_capture.retained;
if (ctx->artifact_available &&
ctx->artifact_capture.retained > before &&
- artifact_write_all(ctx->artifact_fd, chunk,
- (size_t)ctx->artifact_capture.retained - before) == -1) {
- storage_error = 1;
+ (w = artifact_write_all(ctx->artifact_fd, chunk,
+ (size_t)ctx->artifact_capture.retained - before,
+ &deadline)) != ARTIFACT_WRITE_OK) {
+ if (w == ARTIFACT_WRITE_TIMEOUT)
+ timed_out = 1;
+ else if (w == ARTIFACT_WRITE_ERROR)
+ storage_error = 1;
break;
}
if (capture_rc == 1) {
int obj;
*is_error = 0;
+ active_cancelled = 0;
buf_reset(out);
if (ctx->artifact_mode) {
artifact_capture_reset(&ctx->artifact_capture);
ctx->artifact_spool_adopted = 0;
ctx->artifact_spool_detach = 0;
ctx->artifact_result_available = 0;
+ ctx->artifact_job_id = 0;
memset(ctx->artifact_result_id, 0,
sizeof(ctx->artifact_result_id));
ctx->artifact_observed = 0;
blob - 6c12b8371e00e5da0223343f26eeab559cd28c55
blob + 34d0433d53ebf94228f96040f1f521cf5a49d667
--- src/fugu-tool/tools.h
+++ src/fugu-tool/tools.h
int artifact_available;
int artifact_fd;
const char *artifact_id;
+ uint32_t artifact_correlation;
uint64_t artifact_observed;
size_t artifact_raw_len;
enum artifact_termination artifact_termination;
int artifact_spool_detach;
int artifact_result_available;
char artifact_result_id[ARTIFACT_ID_HEX_LEN + 1];
+ int artifact_job_id;
};
struct tool_ctx *tool_ctx_new(int allow_write, int allow_subprocess_net);
/* SIGTERM teardown hook: kill a synchronous subprocess group, if any. */
void tool_cancel_active(void);
+int tool_cancel_requested(void);
/*
* Project context (behavior.md 4), read and written in the executor
