commit 00c243fe224c7d37aeae83f14e4ad33b046fc8c1 from: Isaac Meerleo date: Fri Jul 17 14:00:13 2026 UTC add research notes and feature tickets commit - 7d76e21a04c7df54c9db31a841f6bbc68361030d commit + 00c243fe224c7d37aeae83f14e4ad33b046fc8c1 blob - /dev/null blob + 32801b04ca6157266086737fb4dbc0c864ff02df (mode 644) --- /dev/null +++ .scratch/context-compaction/issues/01-serialize-conversation-for-summarization.md @@ -0,0 +1,84 @@ +# Serialize conversation history for summarization + +Status: needs-triage +Type: task + +## What to build + +Change `/compact` so the summarizer receives one explicitly delimited text +document instead of the original messages in their live User/Assistant roles. +Render the canonical conversation with labels such as: + +```text +[User]: What the Owner said + +[Assistant]: Response text + +[Assistant tool calls]: read(path="foo.c") + +[Tool result]: Tool output +``` + +Wrap that serialization in `` tags and use a summarizer-specific +system instruction that says to summarize, never answer or continue the +embedded conversation. Tool results are capped at 2,000 bytes each and retain +their beginning followed by a conspicuous count of omitted bytes. + +This adapts pi's +[`serializeConversation()`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/compaction/utils.ts) +and summarization request shape rather than copying TypeScript into fugu. The +current fugu message model has no thinking block, so it does not emit pi's +`[Assistant thinking]` record unless that becomes canonical conversation state +in a separate change. + +## Why + +Today `cmd_compact()` passes the active Projection as ordinary provider +messages and appends the summarization instruction as one more User message. +The model can interpret that as a conversation to continue. The raw tool +results also make the compaction request nearly as vulnerable to context +overflow as the request it is meant to rescue. + +## Acceptance criteria + +- [ ] A pure, length-aware serializer handles User text, Assistant text, + Assistant tool calls, and User tool results in Projection order. +- [ ] Labels and blank-line separation are byte-exact and covered by unit + tests; embedded role-looking text remains data inside the delimited document. +- [ ] Each tool result keeps at most 2,000 bytes, never splits valid UTF-8, and + appends an omission marker containing the exact omitted-byte count. +- [ ] Tool names and JSON arguments are rendered deterministically and remain + bounded by the existing Turn/request limits. +- [ ] `/compact` sends exactly one serialized User document under a + summarizer-specific system prompt, with no tools and no live conversation + roles preceding it. +- [ ] The generated summary still becomes the system-block continuation notes, + is journaled, resumes faithfully, and leaves manual `/compact` UI behavior + unchanged. +- [ ] Adversarial tests cover control bytes, malformed UTF-8, empty blocks, + 1,999/2,000/2,001-byte results, multiple tools, and the request-size bound. +- [ ] The compaction contract is documented in `docs/compaction.md` (creating + the document if this is the first compaction ticket implemented). + +## Code-cost analysis + +Expected size: **small-to-medium, about 350-600 lines including tests and +documentation, across 3-5 focused commits**. + +| Seam | Likely work | Cost/risk | +| --- | --- | --- | +| Serializer | New `compaction.c/.h` over `struct msg`/`struct block`, UTF-8-safe truncation, deterministic tool formatting | 140-220 production lines; medium I13 risk | +| Request shape | Let `cmd_compact()` supply a summarizer system prompt and one serialized message instead of the live Projection | 60-120 production lines; medium provider-parity risk | +| Unit and wire tests | New pure suite plus `regress/turn` assertions for exact Anthropic/OpenAI/Claude Subscription bodies | 140-230 test lines | +| Docs/build | Makefile registration and compaction documentation | 30-60 lines | + +The main uncertainty is whether “2 KB” means pi's exact 2,000-character +JavaScript cap or a 2,048-byte fugu cap. This ticket recommends a named +2,000-byte cap so the bound is deterministic in fugu's byte-oriented APIs. + +## Blocked by + +None - can start immediately. + +## Comments + blob - /dev/null blob + 433581e5f0f4776ed170c448fd12264f83095137 (mode 644) --- /dev/null +++ .scratch/context-compaction/issues/02-cumulative-file-tracking-in-summaries.md @@ -0,0 +1,84 @@ +# Carry cumulative file tracking through compaction summaries + +Status: needs-triage +Type: task + +## What to build + +Every fugu-generated compaction summary carries deterministic +`` and `` blocks. Collect paths from structured +`read`, `write`, `edit`, and `multi_edit` tool calls in the span being +summarized, merge them with the previous fugu compaction's structured file +details, sort and deduplicate them, and append the blocks to the model-written +summary before it becomes continuation notes in the system block. + +This follows pi's cumulative file-operation design in +[`compaction.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/compaction/compaction.ts) +and +[`utils.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/compaction/utils.ts): +modified paths win over read-only paths, and prior structured details—not the +model's prose—are the accumulation source. + +## Why + +The information is cheap to derive from tool calls, survives lossy prose +summarization, and is directly useful to the same model when the summary is +injected into fugu's system block. Without cumulative state, a second +compaction can forget files that were important before the first cut. + +## Acceptance criteria + +- [ ] A length-aware collector extracts `path` only from the supported + structured file-tool arguments; shell command text is never guessed as a + path. +- [ ] Read-only paths appear under `` and any path written or + edited appears only under ``, regardless of operation order. +- [ ] Lists are bytewise deterministic, sorted, and deduplicated across the + current span and every prior fugu-generated compaction. +- [ ] The final XML-like blocks are generated by fugu and appended once after + model prose; the model is not trusted to preserve or generate the lists. +- [ ] Paths with newlines, controls, tag delimiters, invalid UTF-8, or excessive + length cannot forge block structure; the encoding/rejection rule is pinned + and tested. +- [ ] The compact journal record stores optional structured `read_files` and + `modified_files` details, remains backward-compatible with old summary-only + records, and restores the sets on resume. +- [ ] `/clear` clears cumulative file state. A failed or cancelled compaction + changes neither the active summary nor its file sets. +- [ ] Two successive `/compact` operations, resume-then-compact, read-then-edit + promotion, duplicates, malformed tool input, and empty lists have regression + coverage. +- [ ] `docs/compaction.md` documents the blocks, cumulative semantics, and that + they describe structured tool calls rather than an audited filesystem log. + +## Code-cost analysis + +Expected size: **medium, about 500-850 lines including tests and docs, across +4-6 focused commits**. + +| Seam | Likely work | Cost/risk | +| --- | --- | --- | +| Collection/formatting | Bounded path sets, JSON argument extraction, modified-wins normalization, safe block rendering | 180-300 production lines; medium untrusted-input risk | +| Durable state | Extend compact journal records and replay with optional arrays; carry/reset state in the coordinator | 100-180 production lines; medium I14 compatibility risk | +| Integration | Merge prior details with the compacted Projection and append generated blocks exactly once | 50-100 production lines | +| Tests/docs | Pure collector tests plus journal, resume, clear, and repeated-compaction integration coverage | 220-350 lines | + +Using optional journal fields costs more than parsing tags back out of prose, +but keeps cumulative state structural and makes malformed model output +irrelevant. It is still a bounded change and introduces no dependency or new +privilege. + +## Blocked by + +None - can start immediately. If ticket 01 is implemented concurrently, land +the shared compaction module first to avoid overlapping edits. + +## Triage decision + +Decide whether a failed file-tool call should be tracked as model-visible work +history (pi tracks calls) or omitted as an unsuccessful operation. The cheap +pi-compatible default is to track accepted tool calls and document that the +lists are not proof that an operation succeeded. + +## Comments + blob - /dev/null blob + fefdaca4e782a2d60e8f50bcc235eadc2d9f7211 (mode 644) --- /dev/null +++ .scratch/context-compaction/issues/03-auto-compaction-with-overflow-recovery.md @@ -0,0 +1,125 @@ +# Add automatic compaction with one-shot overflow recovery + +Status: needs-triage +Type: task + +## What to build + +Add automatic context compaction around the canonical Lead Turn. Trigger it +before a Generation when: + +```text +contextTokens > contextWindow - reserveTokens +``` + +Use defaults of 16,384 reserve tokens and 20,000 recent tokens kept. Select a +cut by walking backward over journal/Projection entries, normally cutting at a +Turn boundary and never at a tool-result message. If one Turn alone exceeds +the keep budget, summarize its early prefix separately and retain a valid +Assistant boundary with every tool call/result pair intact. + +If a provider rejects a request specifically for context overflow, compact the +same Turn state and retry that rejected Generation exactly once. Never rerun an +already executed tool, never retry after accepted content, and never recurse if +the summarization request itself overflows. + +This adopts the high-value behavior documented in pi's +[`docs/compaction.md`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/compaction.md), +adapted to fugu's append-only Journal, active Turn transaction, provider-neutral +Generation, and privilege-separated API worker. + +## Current behavior and implementation constraints + +- `/compact` is manual, requires an idle `turn_txn`, summarizes the entire + Projection, journals one summary string, and resets all messages. +- `/context` has a byte-derived estimate and `effective_window()` can resolve a + configured, provider-reported, or compiled window. There is no tokenizer for + the unsent Projection. +- `turn_run()` treats every provider error as terminal failure. Provider error + type/message currently collapse into a display string, so overflow cannot be + classified safely without extending the Generation/API seam. +- A Lead Turn accepts its User message, Assistant tool calls, and tool results + into an active transaction before commit. Prefix replacement therefore must + preserve `pending_first`, Projection receipts, cache state, and journal + abandonment semantics. +- Resume currently treats a compact record as a full reset. Keeping a recent + suffix requires a durable cut reference or a bounded retained-message + snapshot; prose alone is insufficient. + +## Acceptance criteria + +- [ ] Auto-compaction defaults to enabled and uses named defaults of 16,384 + reserve tokens and 20,000 kept tokens; configuration syntax, validation, and + disable behavior are documented and tested. +- [ ] With a known effective window, the strict `>` threshold is evaluated on + every prospective Lead Generation, including continuations after tool + results. With an unknown window, threshold compaction is skipped but typed + overflow recovery remains available. +- [ ] `contextTokens` uses one documented conservative estimator over the + actual prospective Projection. Provider usage may calibrate it but stale + usage from a prior request is never treated as the exact next-request size. +- [ ] The cut planner normally retains whole recent Turns up to the 20,000-token + target and never starts retained context with a tool-result message or leaves + a tool call without its result. +- [ ] A Turn larger than the keep target is split only at a valid User or + Assistant boundary. Its early prefix receives a distinct Turn-context + summary that is merged with the history summary before the valid suffix. +- [ ] Compaction can replace a prefix while the transaction is idle or active + without committing, abandoning, duplicating, or losing pending Turn state. +- [ ] The compact journal record durably identifies or contains the retained + suffix, and resume reconstructs exactly the same provider-valid Projection. +- [ ] Provider adapters/API IPC expose a bounded typed context-overflow + category based on HTTP status and provider error type/code—not broad English + substring matching. Unknown errors retain today's failure behavior. +- [ ] An overflow-rejected Generation with no accepted content compacts and is + retried once. A second overflow, any post-content failure, cancellation, or + compaction failure ends the Turn conspicuously with no further retry. +- [ ] Tool side effects occur at most once across recovery; tests cover an + overflow after a completed tool-result batch and prove tools are not rerun. +- [ ] Prompt-cache receipts reset after a cut, usage remains additive and + nonduplicated, and the UI distinguishes summarization from the retry. +- [ ] Manual `/compact` remains available and shares the serializer, summary + policy, durable state, and safety checks rather than maintaining a second + implementation. +- [ ] `docs/compaction.md`, `fugu(1)`, `fugu.conf(5)`, behavior, design, and + verification docs define threshold, cuts, split Turns, overflow recovery, + unknown-window behavior, and the exactly-once limit. + +## Code-cost analysis + +Expected size: **medium-large and high-risk, about 1,800-3,000 lines including +tests and documentation, across 7-11 reviewable commits**. The value is high, +but this is not a one-condition change: the cost is transactional correctness +and durable replay. + +| Seam | Likely work | Cost/risk | +| --- | --- | --- | +| Token accounting and cut planner | Prospective Projection estimator, effective-window policy, Turn/tool-pair boundary planner, split-Turn preparation | 220-380 production + 300-500 test lines; medium algorithmic risk | +| Shared compactor | Reuse ticket 01 serialization, summarize history/prefix, merge prior summary/details, enforce no-tools and bounded output | 140-260 production + 180-300 test lines | +| Turn transaction and Journal | Prefix replacement while idle/active, retained-suffix durability, replay, cache receipt reset, clear/abandon behavior | 220-380 production + 350-550 test lines; high I14 risk | +| Provider error typing | Preserve provider error code/type and HTTP classification through `fugu-api` → Generation → Turn result | 120-220 production + 180-300 test lines; medium protocol risk | +| Retry orchestration | Re-enter only the rejected Generation, cap at one, preserve usage and tool exactly-once semantics | 160-300 production + 250-450 integration-test lines; high side-effect risk | +| Config/UI/docs | Defaults, parser/man page, summarize/retry notices, `docs/compaction.md` and verification matrix | 120-220 implementation/doc lines + config tests | + +The estimate assumes tickets 01 and 02 land first. Without them, add roughly +700-1,100 lines and two to four commits. No new runtime dependency or privilege +is required; the security boundary stays the same, but the coordinator and +Journal state machines gain new states that require adversarial crash/resume +tests. + +## Blocked by + +- [01 — Serialize conversation history for summarization](./01-serialize-conversation-for-summarization.md) +- [02 — Carry cumulative file tracking through compaction summaries](./02-cumulative-file-tracking-in-summaries.md) + +## Triage decisions + +1. Pin the configuration names and whether model-specific `context_limit` + remains an override for auto-compaction. +2. Choose the durable suffix representation: stable Journal entry references + or an explicit retained-message snapshot in the compact record. +3. Pin provider-specific overflow codes/statuses from captured primary-source + fixtures before implementing the typed classifier. + +## Comments + blob - /dev/null blob + 1a957c882c49b2701f331f8084e42e1fafef59fd (mode 644) --- /dev/null +++ .scratch/context-epochs/issues/01-evaluate-immutable-context-epochs.md @@ -0,0 +1,95 @@ +# Evaluate immutable Context Epochs and chronological context deltas + +Status: needs-triage +Type: research + +## Research question + +Should fugu stop rebuilding its privileged system block when context changes +and instead persist an immutable Context Epoch plus chronological, model-visible +“context changed” deltas? + +OpenCode V2's +[`specs/v2/session.md`](https://github.com/anomalyco/opencode/blob/dev/specs/v2/session.md) +persists the exact privileged System Context shown to the model as an immutable +baseline. At safe provider-turn boundaries it observes context sources and, if +one changed, atomically journals one chronological System message and advances +the epoch snapshot. A completed compaction or Session move starts a fresh +baseline. This preserves history semantics and allows an unchanged provider +cache prefix to remain valid. + +Fugu currently composes `sysfull` from the base prompt, personal context, +project context, and compaction notes. It journals personal/project bytes and +the summary, rebuilds after startup, `/clear`, and `/compact`, and deliberately +starts cache state cold after `/clear`, `/compact`, `/model`, and resume. Its +canonical conversation has only User and Assistant roles, so chronological +System deltas would be a real model/journal/provider change rather than a +different representation of existing state. + +## What to investigate + +1. Pin the OpenCode source revision and describe Context Epoch admission, + source observation, atomic delta publication, compaction reset, unavailable + source behavior, and prompt-cache intent. +2. Inventory every fugu system-context source and every time it can actually + change during a Session: configured/built-in system prompt, personal and + project context, `#` capture, `/clear`, `/compact`, model/provider selection, + skills, environment/date facts, and resume under changed config/files. +3. Verify or reject the premise that fugu silently rewrites prior context. + `journal_context()` already snapshots project/personal bytes for replay; + determine exactly which baseline inputs remain unjournaled and which live + changes are not re-observed at all. +4. Compare three designs: + - keep full-system rebuilds and document their cache/replay semantics; + - journal full immutable Context Epoch snapshots but still rebuild on an + explicit epoch boundary; or + - add canonical chronological System-delta messages at safe boundaries. +5. For each design, analyze Anthropic and Claude Subscription cache breakpoint + stability, OpenAI behavior, provider request validity, context growth, + compaction interaction, prompt-injection presentation, and crash/resume + fidelity under invariant I14. +6. Measure or estimate whether cache savings justify adding a System role and + context-observation state machine in a one-Owner local process. + +## Deliverable + +A checked-in research note with: + +- a source-by-source current-state and replay table; +- a timeline comparison for startup → context edit → next Turn → compaction → + resume; +- explicit findings for semantic fidelity and provider-cache validity; +- a recommendation to retain, partially adopt, or fully adopt Context Epochs; +- an ADR proposal if the canonical conversation or Journal contract should + change; and +- separately scoped implementation tickets with code-cost estimates only if + adoption is recommended. + +## Acceptance criteria + +- [ ] The mutable OpenCode `dev` source is pinned to a commit. +- [ ] Claims about current fugu behavior cite code, behavior/design docs, and + journal/cache regressions; the ticket's premise is tested, not assumed. +- [ ] The analysis distinguishes a context-source snapshot, a system baseline, + a chronological delta, a compaction summary, and prompt-cache bookkeeping. +- [ ] `/clear`, `/compact`, `#` capture, model switch, changed files before + resume, and changed config before resume each receive an explicit verdict. +- [ ] Any proposal preserves fugu's provider-neutral Projection validity and + append-only crash/replay invariants. +- [ ] The recommendation quantifies implementation complexity and expected + cache benefit well enough to decide whether to build it. + +## Research cost + +Expected size: **medium, roughly 250-500 lines of evidence and design analysis**. +The research is inexpensive; full adoption would likely be medium-to-large +because it touches the canonical message roles, request builders, Journal +replay, context observation, compaction, and all provider cache tests. + +## Blocked by + +None - can start immediately. Coordinate conclusions with the context +compaction tickets because compaction is an epoch boundary in both designs. + +## Comments + blob - /dev/null blob + e61328040cf5009fee5be117039fc35f923c5639 (mode 644) --- /dev/null +++ .scratch/durable-tool-output/issues/01-retain-bounded-tool-output-artifacts.md @@ -0,0 +1,116 @@ +# Retain full tool output as private, bounded artifacts + +Status: needs-triage +Type: task + +## What to build + +Treat output bounding as a settlement boundary. The model still receives a +small bounded preview, but output beyond that preview is retained—up to a +larger hard artifact cap—in Owner-only managed storage. The result names an +opaque output ID, retained byte count, truncation state, and the paged tool the +model can use to inspect another range without rerunning the command. + +Start with foreground `shell`, `grep`, `find`, and `ls`, whose subprocess +capture currently stops at 256 KiB, and `shell_output`, whose 256 KiB ring drops +overflow. Decide explicitly whether brokered web output belongs in the first +slice or a follow-up. + +Do **not** add OpenCode's stale-registration rejection. Fugu's tool definitions +are static C objects selected from startup configuration/skills; it has no +runtime plugin registry, registration generation, or stale executor handle to +reject. Revisit that rule only if dynamic tools/MCP/plugins are introduced. + +## Why the safe version is not just a larger buffer + +- Foreground subprocesses are killed when capture exceeds 256 KiB today, so + bytes beyond the marker do not exist to recover. +- Under `allow_write no`, `fugu-tool` has no `wpath`/`cpath`; creating a file in + the project would weaken the kernel-enforced read-only mode. +- The coordinator can create private 0600 files only inside its existing + session unveil, while the tool worker cannot open that directory. +- A raw session path is not useful to the model because normal file tools + cannot read it, and placing generated logs in the project risks accidental + commits and secret disclosure. +- Truly unbounded retention permits disk exhaustion. “Full” must mean complete + up to a documented per-artifact and aggregate-session quota, followed by a + second conspicuous truncation boundary. + +The preferred architecture is therefore a coordinator-owned artifact store. +The coordinator creates a private file and passes only a write descriptor to +the executing worker; the worker streams to it while constructing the preview. +A provider-visible retrieval tool addresses the artifact by unpredictable ID +and returns bounded offset/limit windows. No role receives a wider path unveil. + +## Acceptance criteria + +- [ ] Tool results return a bounded, useful preview and structured metadata: + opaque ID, total bytes observed, retained bytes, whether the preview was + abbreviated, and whether the artifact hard cap was reached. +- [ ] Preview policy is pinned (for example useful head and tail regions) and + remains within the existing per-tool/Turn model-context bounds. +- [ ] A new paged retrieval tool reads an artifact by ID with byte or line + offset/limit, reports the next cursor and total retained size, and never + accepts a filesystem path. +- [ ] Artifact IDs are unguessable within the process, length-checked, bound to + the current Session, and cannot traverse or alias another Owner file. +- [ ] Files and directories are Owner-only (0600/0700), created atomically + under the existing session authority, and never placed in the project tree. +- [ ] The executor receives only pre-opened descriptors. `allow_write no`, + subagent read-only confinement, API/web no-file authority, and every current + unveil set remain unchanged and are proved by sandbox regressions. +- [ ] Foreground commands continue after the 256 KiB preview boundary but are + killed at timeout or the larger artifact cap. Disk writes and IPC draining + are bounded so a fast producer cannot exhaust memory or starve the event + loop. +- [ ] Background jobs spool incrementally instead of silently dropping ring + overflow; polling remains bounded and does not replay bytes already returned. +- [ ] Artifact creation/finalization and the corresponding tool-result + settlement have defined crash behavior. A failed or abandoned Turn cannot + leave a journaled ID that points to another Turn's data. +- [ ] Resume either preserves journaled artifact IDs for paged retrieval or + marks them durably unavailable; the chosen retention and cleanup policy is + explicit and tested. +- [ ] Per-artifact, per-Session, and global cleanup/quota behavior prevents disk + exhaustion and handles a full filesystem conspicuously without losing the + bounded model result. +- [ ] Tests cover exact preview boundaries, multi-megabyte output, binary/NUL + bytes, invalid IDs/cursors, timeout, hard-cap termination, cancellation, + crash/resume, read-only mode, concurrent subagents/jobs, and cleanup. +- [ ] Behavior, security caveats, tool descriptions, manuals, and the + verification matrix explain that retained output may contain secrets and is + private Session state. + +## Code-cost analysis + +Expected size: **medium-large, roughly 1,000-1,800 lines including tests and +documentation, across 6-9 commits**. A project-local tempfile would be much +smaller, but would fail under `allow_write no`, pollute the working tree, and +weaken fugu's authority model. + +| Seam | Likely work | Cost/risk | +| --- | --- | --- | +| Artifact store | ID generation, private files, quotas, finalize/reopen/cleanup, crash behavior | 180-320 production lines; high disk/state risk | +| Descriptor protocol | Coordinator creates/passes fds; tool worker streams without new unveil rights | 100-200 production lines; medium IPC risk | +| Capture and jobs | Tee subprocess output to artifact while retaining bounded preview; replace lossy job ring semantics | 180-320 production lines; high backpressure/process risk | +| Retrieval tool | Definition, argument validation, bounded paging, Session lookup | 120-220 production lines | +| Tests/docs | Pure store tests, tool/Turn integration, sandbox/ktrace, resume/quota adversarial cases, manuals | 400-700 lines | + +## Blocked by + +None - can start after triage, but storage lifetime and quota choices must be +pinned before implementation. + +## Triage decisions + +1. Per-artifact and per-Session hard caps, and whether hitting the hard cap + kills the producer or continues while discarding. +2. Preview shape: head only, head+tail, or tool-specific. +3. Artifact lifetime across successful, failed, abandoned, and resumed Turns. +4. Whether the first slice covers web responses and subagent-internal tools or + only Lead local subprocess tools/background jobs. +5. Whether Owner-facing output exposes the private path while the model sees + only the opaque ID. + +## Comments + blob - /dev/null blob + 40b2055ad61274f3a3f3f657a596f1e53de365a7 (mode 644) --- /dev/null +++ .scratch/model-thinking-effort/issues/01-configurable-thinking-effort.md @@ -0,0 +1,59 @@ +# Configurable model thinking effort + +Status: needs-triage +Type: task + +## Problem + +fugu has no way to select a model's thinking or reasoning effort. The +configuration carries the model, output-token cap, and context-window override, +but no effort setting. The Anthropic, OpenAI, and experimental Claude +Subscription request builders therefore rely entirely on provider defaults. + +This is especially visible with models that expose named levels such as +`low`, `medium`, `high`, and `xhigh`: changing models may silently change the +amount of reasoning because fugu cannot preserve an explicit Owner choice. + +## Evidence + +- `struct fugu_conf` and the parser have no effort field or directive. +- `anthropic_build_request()`, `openai_build_request()`, and + `claude_sub_build_request()` emit no thinking/reasoning effort parameter. +- The experimental Claude Subscription compatibility profile advertises an + effort beta, but its request adapter does not currently use it. + +## Desired behavior + +Add an optional symbolic effort setting that can express the levels supported +by the selected model/provider. Leaving it unset must preserve today's +provider-default behavior. + +The selected value must reach every Generation consistently, including tool +continuations and compaction. The design must explicitly decide whether +subagents inherit the Lead setting or receive a separate override. + +## Acceptance criteria + +- Configuration accepts and validates an effort directive with a documented + set of symbolic values. +- `fugu -n` prints the effective nonsecret value. +- Each supported provider adapter maps the symbolic value to its native wire + field; an unsupported provider/model combination fails conspicuously or has + a documented fallback rather than silently selecting a different level. +- The value survives model selection and every Generation within a Turn. +- Subagent inheritance or override behavior is explicit and covered by tests. +- Request-builder regressions assert the exact wire shape for every supported + provider, and config regressions cover valid, invalid, default, and match + precedence behavior. +- `fugu.conf(5)` documents values, defaults, provider limitations, and examples. + +## Triage decisions needed + +1. Directive name: `effort`, `thinking_effort`, or another term. +2. Canonical value set and how provider-specific names such as `xhigh`/`max` + map without pretending unsupported levels are equivalent. +3. Whether subagents inherit the Lead value or gain `subagent_effort`. +4. Whether unsupported model/provider combinations are rejected locally or + passed through for the provider to reject. + +## Comments blob - /dev/null blob + b1cbc5ef5c62127cf30b88365543dacc7803da60 (mode 644) --- /dev/null +++ .scratch/prompt-admission-semantics/issues/01-compare-steer-and-queue-promotion-rules.md @@ -0,0 +1,85 @@ +# Compare steer and queue promotion rules with OpenCode V2 + +Status: needs-triage +Type: research + +## Research question + +Does fugu's specification completely define when queued and steered User input +becomes model-visible, especially when multiple inputs arrive around tool +continuations and the per-Turn tool allowance? + +OpenCode V2 separates durable admission (`PromptAdmitted`) from atomic +promotion into model-visible history (`Prompted`). Fugu already has a cleaner +single-process analogue: the tty queue is pending interface state, steers enter +the active Turn transaction only at a safe tool-result boundary, and the +Pending Projection becomes durable only when the Turn commits. There is no +structural inbox architecture to adopt. + +The useful comparison is OpenCode's more explicit promotion policy in +[`specs/v2/session.md`](https://github.com/anomalyco/opencode/blob/dev/specs/v2/session.md): + +- steers promote at the next safe provider-turn boundary; +- queues remain FIFO while the current drain requires continuation; +- when execution would become idle, promote exactly one queued input and then + reevaluate continuation before promoting another; and +- multiple steers promoted at one boundary reset the provider-turn allowance + once, not once per steer. + +## What to investigate + +1. Pin the OpenCode source revision and restate its `PromptAdmitted` → + `Prompted`, steer, queue, and allowance rules as a small state machine. +2. Trace fugu's actual behavior through `fugu-tty` queue ownership, + `UI_STEER`, `coord.c`'s aggregated `steer` buffer, `turn_mechanics`, the + Turn transaction, Journal commit/abandon, and resume. +3. Compare specification, implementation, and regressions for at least: + multiple steers before one tool-result boundary; a steer after the final + provider response; two or more FIFO queue entries; a queued prompt whose + Turn itself continues through tools; interleaved queue/steer input; + failure/cancellation/session switch; and resume. +4. Determine what “reset the Turn allowance once” should mean in fugu, whose + 50-step tool limit currently belongs to one canonical Turn. Do not assume + OpenCode's counter has identical semantics. +5. Identify specification gaps separately from implementation bugs. Do not + propose a durable inbox or multi-process event architecture unless a + concrete fugu requirement needs one. + +## Deliverable + +A checked-in research note containing: + +- an OpenCode-to-fugu state/transition table; +- exact evidence links and fugu file/line references; +- a verdict for each OpenCode rule: already guaranteed, implemented but + underspecified, missing test, or behavior change needed; +- proposed replacement wording for `handoff/behavior.md` and + `handoff/verification.md` where precision is missing; and +- separate implementation tickets only for confirmed code changes. + +## Acceptance criteria + +- [ ] The OpenCode commit is pinned; mutable `dev` documentation is not treated + as timeless evidence. +- [ ] The analysis distinguishes admission, promotion, provider dispatch, + Journal acceptance, Turn commit, and UI dequeue. +- [ ] “Promote one queue entry, then reevaluate” and “one allowance reset per + steer batch” each receive an explicit fugu verdict. +- [ ] Existing regressions are mapped to the edge cases they prove, and missing + cases are named with proposed fixtures/assertions. +- [ ] The recommendation preserves fugu's one-Owner, one-process queue design + unless evidence demonstrates a real durability requirement. + +## Research cost + +Expected size: **small, one focused research pass and roughly 150-300 lines of +notes/spec proposals**. If the implementation matches but the contract is +ambiguous, the follow-up should be docs plus targeted regression cases rather +than a runtime refactor. + +## Blocked by + +None - can start immediately. + +## Comments + blob - /dev/null blob + 81466145e17b8abac8081fd5ac06ed70c54b801b (mode 644) --- /dev/null +++ .scratch/user-message-background/issues/01-grey-background-for-user-transcript-messages.md @@ -0,0 +1,61 @@ +# Render transcript User messages on a grey background + +Status: needs-triage +Type: task + +## What to build + +Give each User message in the curses transcript a grey-background block so it +is visually distinct from light-grey Assistant prose and indented cyan tool +activity. Reuse the same palette/treatment intended for the User input area so +composing and reviewing the Owner's words feel like one visual language. + +The block must include the `> ` prefix, wrapped continuation lines, and padding +to a deliberate edge. It must not recolor Assistant output or change line and +print modes. + +## Acceptance criteria + +- [ ] Color terminals render every `IT_USER` transcript item with the chosen + grey background and readable foreground contrast. +- [ ] All visual rows belonging to one wrapped or multiline User message have + coherent background fill; no unpainted gaps appear between the prefix, text, + continuation indent, and chosen right edge. +- [ ] The treatment shares a named color-pair/palette helper with the input + area rather than duplicating numeric curses attributes. +- [ ] Monochrome terminals retain a clear, readable fallback (for example + bold/reverse) without relying on color. +- [ ] Narrow terminals, wide/combining UTF-8, `ascii yes`, resize/reflow, + scrollback, and the conversation drawer preserve correct geometry. +- [ ] Headless renderer tests pin attributes and wrapping, and the pty + regression proves terminal startup/teardown remains clean. +- [ ] `handoff/behavior.md`, `docs/design/m7-curses.md`, and `fugu(1)` describe + User transcript styling without pinning an implementation-only color number. + +## Code-cost analysis + +Expected size: **small, about 120-240 lines including tests/docs, in 2-3 +commits**. The text attribute is easy; the real cost is painting padding across +every wrapped display row without bleeding the background into adjacent +transcript items. + +Likely touch points are `src/fugu-tty/ui.c`, a render-focused regression (or a +small extraction that makes span attributes testable), the curses pty check, +and the three interface documents. + +## Blocked by + +None - can start immediately. + +## Triage decisions + +1. Should the block fill the transcript width, the content width plus padding, + or a fixed percentage/maximum width? +2. Is “grey box used for User input” already a desired-but-unimplemented style + (the current `draw_input()` has no background pair), in which case this + ticket should style both surfaces together? +3. Pick foreground/background values for 8-color terminals and define the + monochrome fallback. + +## Comments + blob - /dev/null blob + 2af38effe39ffa1982e46c542826e56f3d56f05c (mode 644) --- /dev/null +++ docs/research/pi-supply-chain-posture.md @@ -0,0 +1,275 @@ +# Pi supply-chain posture: what fugu should adopt + +Research date: 2026-07-16 + +Pi source inspected: the current canonical `earendil-works/pi` repository at +commit +[`216e672`](https://github.com/earendil-works/pi/tree/216e672e7c9fc65682553394b74e483c0c9e47f7). +The former `badlogic/pi-mono` URL redirects there. + +## Recommendation + +Do **not** copy Pi's npm controls wholesale. They are good controls for Pi's +large, registry-resolved Node dependency graph, but fugu has deliberately +removed that attack surface: it links only OpenBSD base libraries, has no +language-registry dependency resolution step, and carries one small source +dependency in the repository. + +Adopt two ideas in forms native to fugu: + +1. **Adapt exact pinning into explicit vendored-source provenance.** Record the + upstream URL, commit, and digest for `src/common/jsmn.h`, and optionally + verify the digest in `make check`. The file is already fixed by the fugu Git + commit; the missing part is an auditable upstream identity. +2. **Adopt the outside-the-repository artifact test in `make live-check`.** + Fugu already does most of this more strongly than Pi by building an OpenBSD + package and installing it on a clean host. Make the artifact-only property + explicit by running the installed binary from a temporary project outside + the checkout, with development worker/config overrides unavailable. + +Reject `min-release-age`, `--ignore-scripts`, and a lifecycle-script allowlist +for the current tree. They control registry selection and package-manager +execution that do not occur in fugu. Reconsider them only if fugu later adds a +registry dependency, ports library dependency, language runtime, generated +foreign build step, or plugin/package ecosystem. + +## Decision matrix + +| Pi control | Decision for fugu | Reason | Estimated cost | +|---|---|---|---| +| Exact-pin dependencies | **Adapt; mostly satisfied already** | Committed vendored source and checksummed release distfiles are immutable inputs, but jsmn's upstream commit/digest is not recorded | Low: documentation alone is under an hour; an enforced digest/provenance regression is roughly 20-60 lines and a few hours including tests | +| Minimum release age | **Reject now** | No registry resolver selects fresh releases during a fugu build or install | No implementation cost; a custom age mechanism would add maintenance without covering a current path | +| `--ignore-scripts` everywhere | **Reject as Node-specific** | No npm lifecycle scripts execute; BSD make and ports hooks are first-party, reviewed build instructions and cannot simply be disabled | No implementation cost; preserve the existing no-runtime/no-ports-dependency charter | +| Lifecycle-script allowlist | **Reject now; add only with such dependencies** | There is no lifecycle-script metadata or dependency graph to inspect | No implementation cost today; a premature checker would be policy theatre | +| Release install outside the repo | **Adopt and strengthen `make live-check`** | Detects source-tree fallback, missing package files, wrong installed paths, and environment leakage | Low-medium: about half to one day for shell changes, regression coverage, documentation, and one clean-host validation; no new runtime dependency | + +The estimates assume the current OpenBSD-only architecture and include tests +and documentation, but not release-machine provisioning. + +## What Pi actually implements + +Pi's stated policy is that npm dependency changes are reviewed code changes. +Its current README lists exact direct dependency versions, a lockfile and +published shrinkwrap, a two-day release-age gate, lifecycle-script suppression, +a lifecycle-script review list, audits, and isolated local release installs. +See the +[`Supply-chain hardening` section](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/README.md#L61-L73). + +### Exact direct and transitive versions + +Pi sets `save-exact=true` and checks every external entry in +`dependencies`, `devDependencies`, and `optionalDependencies` against an exact +semantic-version pattern. Workspace and non-registry references are treated +separately. The enforcement is source-visible in +[`check-pinned-deps.mjs`](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/check-pinned-deps.mjs#L1-L63), +not just a README convention. + +The root lockfile fixes development resolution, while the published coding +agent carries a generated `npm-shrinkwrap.json` for its transitive production +closure. This matters for Pi because an exact direct dependency can still +declare ranged transitive dependencies. Fugu has no corresponding resolver or +transitive package graph. + +### Two-day release cooldown + +Pi's +[`.npmrc`](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/.npmrc#L1-L2) +sets `min-release-age=2`. npm defines that value in days: versions newer than +the window are excluded from resolution. npm also documents the tradeoff that +the gate can temporarily block a newly published security fix, requiring an +exception or a relaxed window; see the official +[`min-release-age` configuration](https://docs.npmjs.com/cli/v11/commands/npm-install/#min-release-age). + +This is a quarantine control, not integrity verification. It gives registry +and community detection systems time to react to a malicious new publication, +but it neither proves that an older version is safe nor protects a version +already fixed in a lockfile from a compromised download channel. + +### Lifecycle scripts: suppression plus review + +Pi's development, CI, local-release install, publish, and self-update paths use +`--ignore-scripts` where supported. The policy is visible in the +[CI install](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/.github/workflows/ci.yml#L32-L39), +[publish path](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/publish.mjs#L52-L53), +and +[self-update construction](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/packages/coding-agent/src/config.ts#L115-L182), +as well as the local-release source below. npm defines that flag as suppressing +scripts from package manifests while still allowing an explicitly requested +`npm run` command; see npm's +[`ignore-scripts` semantics](https://docs.npmjs.com/cli/v11/commands/npm-install/#ignore-scripts). +This removes the install-time code-execution path used by malicious +`preinstall`, `install`, and `postinstall` scripts. + +Pi also inspects the generated production shrinkwrap for packages marked +`hasInstallScript`. The current allowlist names two exact package versions and +records why their scripts are considered harmless; any new marked package +fails generation until reviewed. See the +[`allowedInstallScriptPackages` list](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/generate-coding-agent-shrinkwrap.mjs#L7-L16) +and the +[`hasInstallScript` validation](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/generate-coding-agent-shrinkwrap.mjs#L224-L260). +In Pi this is primarily an audit gate: normal install paths still suppress the +scripts. It ensures that a dependency cannot silently acquire script-bearing +metadata without review. + +### The outside-repository release check + +Pi's `release:local` script requires its output directory to be outside the +repository, runs checks and tests, rebuilds publishable packages, packs them, +and creates fresh npm and Bun production installs from those tarballs. See +[`prepareOutputDirectory`](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/local-release.mjs#L107-L132) +and the +[`pack and isolated-install sequence`](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/local-release.mjs#L197-L255). + +One nuance matters when copying the pattern: the current script creates the +isolated installs and prints outside-repo `pi --help` commands, but does not +itself execute those commands. See its +[`final smoke instructions`](https://github.com/earendil-works/pi/blob/216e672e7c9fc65682553394b74e483c0c9e47f7/scripts/local-release.mjs#L259-L283). +The useful control is therefore artifact isolation; fugu can and should make +the actual invocation automatic. + +## Fugu's current posture + +### Dependency surface + +Fugu's [dependency charter](../../handoff/charter.md) permits only OpenBSD base +libraries in linked binaries and permits a small, permissively licensed, +single-file vendored component only where base has no safe equivalent. The +[port Makefile](../../port/productivity/fugu/Makefile) declares no +`BUILD_DEPENDS`, `RUN_DEPENDS`, `LIB_DEPENDS`, `TEST_DEPENDS`, or `MODULES`; its +only dependency declaration is the base-library `WANTLIB` set +`c curses event tls util`. The +[deployment regression](../../regress/deploy/deploy_test.sh) derives each +binary's actual ELF dependencies and checks that their union exactly matches +that set. + +The sole external source component is `src/common/jsmn.h`, documented in the +[README](../../README) and isolated behind `src/common/json.c`. It is committed +source, so every fugu commit already pins its exact bytes. At the research +snapshot its SHA-256 is +`c04533e9181e1e33baceb0f55ac449b05145bb936e8c68cc77dfe0d8277514fb`, +and it is byte-identical to upstream jsmn at +[`25647e6`](https://github.com/zserge/jsmn/blob/25647e692c7906b96ffd2b05ca54c097948e879c/jsmn.h). +Neither the upstream commit nor digest is currently recorded in the tree. That +is a traceability gap, not dynamic dependency drift. + +### Source and package integrity + +[`make local-port`](../../scripts/local-port) refuses a dirty worktree, archives +the exact current Git commit, stages the maintained port under `mystuff`, and +runs the ports framework's `makesum` and `checksum` targets. OpenBSD specifies +that `checksum` compares every fetched distfile and patch against the SHA-256 +values in `distinfo`; see +[`bsd.port.mk(5)`](https://man.openbsd.org/OpenBSD-7.9/bsd.port.mk.5#checksum). +The OpenBSD porting guide also requires visible source versions rather than a +moving `latest` file and recorded checksums for retrieved sources; see the +official [Porting Guide](https://www.openbsd.org/faq/ports/guide.html#PortsChecklist). + +This is the relevant equivalent of lockfile integrity for fugu's own release +distfile. It does not authenticate a malicious commit accepted into the fugu +repository, but neither do Pi's exact version strings; source review and +repository trust remain separate controls. + +### Build and release isolation + +[`make check`](../../scripts/check) removes prior build products, creates fresh +object directories, compiles with warnings treated as failures, and runs every +regression suite. The ports `package` target then builds from the checksummed +archive through extraction, fake installation, and package creation. OpenBSD's +framework verifies declared dependencies and `WANTLIB` before building and +can check the final package's library declarations; see +[`bsd.port.mk(5)` dependency and package targets](https://man.openbsd.org/OpenBSD-7.9/bsd.port.mk.5#prepare). +The official testing guide additionally warns against accidentally detecting +undeclared software already present on the build host; see the +[OpenBSD Ports Testing Guide](https://www.openbsd.org/faq/ports/testing.html#Testing). + +Fugu's [`make live-check-privileged`](../../scripts/live-check) is already +stronger than Pi's isolated install step in several respects. On a disposable +host it rejects a pre-existing fugu package, binary, group, or configuration; +snapshots and hashes the package; requires a hostname-plus-digest confirmation; +installs with `pkg_add`; verifies package registration, ownership, modes, and +files; and invokes the installed `fugu -n` as an ordinary user. The companion +[release procedure](../release-live-checks.md) keeps credentialed model tests +separate from privileged package installation. + +The remaining isolation gap is small but real: the privileged invocation +inherits the directory from which the make target was launched, while the +credentialed target defaults some invocations back to the fugu repository. +Several tool and subagent checks already use a temporary project, but the +artifact-only property is not a single enforced invariant. A packaged release +could therefore be insufficiently tested for a source-tree fallback or an +undeclared runtime file. + +OpenBSD package signing is a separate, ecosystem-native concern. Official +packages are signed with `signify`, and OpenBSD documents how maintainers can +sign their own packages in the +[Package Signatures guide](https://www.openbsd.org/faq/ports/ports.html#PkgSig). +Fugu's current clean-host check binds installation to an operator-confirmed +package SHA-256, which is suitable for the present local one-Owner flow. If +packages are later distributed across trust boundaries, signing them would be +more relevant than importing npm release-age or lifecycle controls. + +## Threat and cost analysis + +Pi's controls address three high-probability Node threats: + +- semver or transitive resolution selecting an attacker-controlled registry + publication; +- package lifecycle code executing before the dependency is reviewed; and +- a published tarball working only because it can see monorepo files or a + hoisted development dependency. + +For fugu, the first two paths are absent. The comparable current threats are: + +- an unreviewed change to the committed jsmn bytes or loss of its upstream + provenance; +- a release archive differing from the reviewed commit; +- a package omitting a worker, manual, sample, permission, or other runtime + resource; and +- an installed binary accidentally relying on the source checkout, development + environment variables, or build-host state. + +The current Git, ports checksum, package, ELF dependency, and clean-host checks +already cover most of these. That is why adding npm-shaped controls would add +more policy surface than protection. + +The two recommended adaptations have favorable cost-to-value ratios: + +- **Vendored provenance:** no runtime or build dependency, negligible ongoing + cost, and a direct audit trail for the only imported source. A digest check + should fail only when that file is intentionally reviewed and updated. +- **Artifact-only live smoke:** reuses the existing temporary project, + installed package, hostile development-override checks, and clean-host + harness. Its release-time cost is seconds for `-n`/startup probes, plus any + already-authorized provider quota if the credentialed phase is included. It + catches a class of packaging errors that source regressions cannot. + +By contrast, a minimum-age database for one manually updated header would need +timestamps and exception policy while providing no download-time enforcement. +An ignore/allow-script layer would have nothing to inspect and could encourage +confusing normal, reviewed OpenBSD port hooks with third-party package +lifecycle execution. Both should remain explicit reconsideration triggers, +not current implementation work. + +## Concrete release-policy shape + +Without adding a new dependency system, the eventual fugu policy should be: + +1. Every vendored source file names its upstream repository, immutable commit, + license, and digest; an intentional update changes those facts in the same + reviewed commit. +2. Release source is produced only from clean committed Git state and accepted + by the ports checksum target. +3. The package is built through the ports fake/package stages and its actual + linkage and packing list are checked. +4. `make live-check` runs the installed front end from a newly created project + outside the source checkout, with development path overrides absent or + deliberately poisoned, and fails if any checkout resource is required. +5. The privileged clean-host and ordinary-user credentialed phases remain + separate; this artifact smoke does not move network credentials into a root + process. +6. Registry cooldowns and lifecycle-script policy become mandatory design + questions only if a future change introduces a registry resolver or foreign + install-time executable code. + +This preserves fugu's smaller supply chain instead of recreating Pi's larger +one merely to apply Pi's mitigations.